www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - D project management for better growth and longevity

reply "js.mdnq" <js_adddot+mdng gmail.com> writes:
This is an idea I have that I believe can help D become in time 
one of the most popular, if not the, programming language.

The D language, excluding some of known issues, has many great 
language constructs that set it above the mainstream programming 
languages today. While there are many competitors to D, D seems 
to have stronger fundamentals and IMO a better language syntax 
steaming from it's relation to C/C++.

There are several problems with D as it stands but the main one 
is the userbase. For proper development there must be users or 
are willing to invest the time and energy to create the tools to 
make more users want to use D(which will result in a cycle (or 
rather a spiral)).

D needs a lot of work, more importantly, D development needs a 
structured approach to hardness it's development. I feel D's 
development is sporadic and many user created tools and resources 
are localized, hard to use, etc...

So, what I propose is that D use a sort of integrated project 
development application(web based) that allows users to easily 
help in D's development. It is difficult to explain exactly what 
I'm talking about because I do not know the terms to use or have 
a really clear idea myself.

Therefor, it would be better for me to explain what a user my 
experience:

Suppose I am interested in D. I download the compiler play around 
with it and find some bug. For argument sake, suppose I want to 
improve upon the GC and I have also written a tool that parses D 
code and changes UDA's.

But suppose I could log into to D's web site and access a web 
page that gave an overview of all the develop that D is going 
through. I could see a graphical chart, click on each graphic 
that will take me further into the development of D(more specific 
areas) which allow me to see who is working on what, what has 
been done, and communicate with others on the specifics of that 
topic.

e.g., I want to write a better GC. So I go to the development 
chart find where the GC development area is, and click on it. I 
log in, sign up, and discuss or download what others have done in 
this area. I can sign up to help others in this area, etc...

Hopefully you guys get the picture. The idea is to make it really 
easy to develop for D and the development chart would be massive. 
IDE's, VM's, D lang specifics, tools, documentation, modules, 
etc...

One could also integrate it with GIT so pull's could be 
"visualized" and it would be easier for the project leaders to 
get and send pulls. (for example, there could be several GC 
projects on the GC area)

The goal is to facilitate users to develop D quickly and 
efficiently. It makes it easier to find other users to develop 
"projects", get information(a sort of wiki for each "node" in the 
tree), and pull in the projects into D's main repository.


D
   Language
     Specification
       ...
     Documentation
       ...
     2.x
       ...
     3.x
       ...
   IDE
       ...
   VM
       ...
   Tools
       ...
   Modules
       ...
   etc...

I envision one day when D could be used across many 
platforms(common and not so common) very easily. Say a single 
unified IDE that can be used to develop across multiple platform, 
a great debugger, many integrated tools, build in gui 
development, etc... (basically a one stop shop)

(a script would be used that would do something which also runs a 
script on each child. This allows one to "compile" the hierarchy 
quite easily which would also include builds of the D language, 
modules, and tools. When one installs D a similar hierarchy would 
be presented above allowing them to "install" the modules node(or 
select specifics in the sub node), etc)




Each node in the above tree is backed by a git repository, a wiki 
like helps system, a forum(and each super node's forum includes 
all child forums), and project management stuff.

The owner of D, can, for example, pull in any changes to all it's 
subnodes, and each parent can pull in the immediate children. 
Essentially any modifications work there way up from the leafs to 
the branches but must be verified by allowed by the parent.

Anyways, it's just an idea I came up with. The point is not so 
much that it's new but about combining all the stuff out there in 
a very good way to deal with the extremely large project 
development cycle needed for D to be big(or any project).

It's number one goal is to make it easy for users to contribute 
and to give users to contribute something. Example: someone could 
put in a request for some tool at some node(say you are looking 
for some tool and can't find it then you can make a request) 
which give the ability for someone else to write one and it will 
be added as a node/leaf(which someone could fork).
Feb 21 2013
next sibling parent reply "bearophile" <bearophileHUGS lycos.com> writes:
See also Gerrit, etc:

http://forum.dlang.org/thread/kzjhkzskghqopvcnacla forum.dlang.org

Bye,
bearophile
Feb 21 2013
next sibling parent reply "qznc" <qznc go.to> writes:
On Thursday, 21 February 2013 at 14:29:26 UTC, bearophile wrote:

 See also Gerrit, etc:

 http://forum.dlang.org/thread/kzjhkzskghqopvcnacla forum.dlang.org


Github is not enough?

I have no practical experience with Gerrit. What does it offer 
that Github does not?
Feb 21 2013
next sibling parent reply "bearophile" <bearophileHUGS lycos.com> writes:
qznc:


 I have no practical experience with Gerrit. What does it offer 
 that Github does not?


The answer to your question is in my post and in two links I've 
provided there.

Bye,
bearophile
Feb 21 2013
parent "qznc" <qznc go.to> writes:
On Thursday, 21 February 2013 at 15:32:02 UTC, bearophile wrote:

 qznc:


 I have no practical experience with Gerrit. What does it offer 
 that Github does not?


 The answer to your question is in my post and in two links I've 
 provided there.


I'm too stupid to read the answer from between the lines. ;)

If I understand correctly: Gerrit speeds up the review process. 
D's review process is working fine, the merge is the bottleneck.

Is that really the case? The number of open pull requests seems 
to relatively stable at around 100 for at least two years[0]. An 
open pull request can mean both "needs review" and "needs merge".

Do we need more people to provide pull requests? More people 
reviewing pull requests? Improve approval/merge speed?


[0] 
http://forum.dlang.org/thread/hllgumyacmdfmzqjejfv forum.dlang.org
Feb 21 2013
prev sibling parent "js.mdnq" <js_adddot+mdng gmail.com> writes:
On Thursday, 21 February 2013 at 15:25:56 UTC, qznc wrote:

 On Thursday, 21 February 2013 at 14:29:26 UTC, bearophile wrote:

 See also Gerrit, etc:

 http://forum.dlang.org/thread/kzjhkzskghqopvcnacla forum.dlang.org


 Github is not enough?

 I have no practical experience with Gerrit. What does it offer 
 that Github does not?


Github's forks have no user relationship between forks. Each fork 
is logically seperated from every other in the sense I am talking 
about.

I'm visualizing a whole front end and back end system that 
supports development and project management not just a way to 
version projects.

The goal is to make it very easy for users to contribute. The 
easier it is the more users will contribute AND the more 
productive they will be.

For example, suppose I write my GC. How does that related to D? 
Sure I can push the changes to D's git repository but how does it 
relate? How do I communicate with those people that I am pushing 
to? How do I communicate with others working on similar 
projects(maybe I want to merge)? It is not easy with git. There 
is no real hierarchy involved exist the physical forking. Sure 
you can get emails of people or all get on the same forum but 
then you are dealing with non-related topics... sure you can 
create your own forum or have a sub-forum... but again, it isn't 
streamlined and hierarchical.

As I said, git could be used for each project but a front end 
would overlay it to "connect the dots" and be able to visualize 
the hierarchy and delve into all the projects, and when you 
contribute your own project it is integrated into the system 
naturally. Any pushes can be setup to automatically build into 
the system. (so if you make a GC one can setup the system to 
automatically compile it into D or if you make some tool it can 
automatically include it into the ide interface and/or installer)
Feb 21 2013
prev sibling parent "js.mdnq" <js_adddot+mdng gmail.com> writes:
On Thursday, 21 February 2013 at 14:29:26 UTC, bearophile wrote:

 See also Gerrit, etc:

 http://forum.dlang.org/thread/kzjhkzskghqopvcnacla forum.dlang.org

 Bye,
 bearophile


This is just a forum. There is no hierarchical relationship 
between projects and user collaboration is hard to 
initiate(requires making a post and get emails, etc...). One 
can't work themselves down into the project hierarchy to get to 
more specific details.

An IDE, for example, would compose many areas

IDE - IDE's for D
   DIDE - The D IDE
      Main - The main project
        Core - The core code
        Gui -
        ....
      Debugger -
        ....
      Plugins -
        ....
   D for Eclipse - An D plugin for Eclipse
      ....
   ....

A person would normally see for the overview, just IDE(in this 
case). If they want to download or contribute to the IDE 
development they will follow the IDE link which will expand the 
children of IDE. This allows them to see currently what kind of 
IDE development is taking place. There will be information 
supplied about who to contact, current stats, a forum 
specifically for IDE discussion, etc. If they are interested in 
DIDE they can click on it and get the same type of 
information(but specific to DIDE). If they want to see if a 
specific type of plugin is under development or done they can 
click on plugins. If they want to write a new plugin they have 
the ability to start a new plugin and it will be added.

As you descend down the tree the project development becomes more 
specific all higher levels have access to the lower 
levels(essentially one can view the D forum which will show the 
forums for all the lower levels(the children)).

Such a hierarchical design breaks down the project into as small 
pieces as required. For example, further down the chain one could 
have a specific node representing a single function that is used. 
A complete discussion about the function and it's behavior can be 
found in it's forum and who is monitoring and working on it can 
be communicated with. Its forum will not show any messages from 
other nodes(since it is a terminal in the tree). (but any 
ancestor will have access to it's forum and information)

For example, suppose you write your nifty GC but there is a 
problem with a function. Someone can "clone" just the function 
and write there own/fix up the problem and you can merge it if 
you feel like it(and are the owner of the project).

So, the idea is to have macroscopic level and microscopic.(but it 
would all be uniformly the same).
Feb 21 2013
prev sibling next sibling parent reply Samuel Lampa <samuel.lampa gmail.com> writes:
I think I might see something of what you mean.

For example, the highly successful (not the least in terms of ecosystem) 
Drupal project, has many subprojects, for improving things around Drupal 
(specific set of modules/plug-ins/project infrastructure and many other 
things), which are organised as groups on their group-discussion site 
(powered by the "organic groups" drupal module, which is highly suited 
for this kind of self-organizing groups).

http://groups.drupal.org/

(Anyone can start a new group, to facilitate community self-organization)

You mentioned the main criteria, to get something such to work. which 
not many systems have, namely making it easy to keeping track of who is 
interested/involved in what, in order to efficiently interact with the 
relevant sub-group of the community.

What D has in this area, seems to be the DSource website, and forums:
http://www.dsource.org/forums/

I don't know how it works out though (haven't participated), but it 
turns out that the above feature is missing: It's not easy to keep track 
of who are involved (subscribe to) in projects. Also, it could maybe be 
made easier to find for newcomers?

(Maybe Drupal with "organic groups" could be used to set up an improved 
group discussion site? :) )

Cheers
// Samuel
Feb 21 2013
parent "js.mdnq" <js_adddot+mdng gmail.com> writes:
On Thursday, 21 February 2013 at 15:12:40 UTC, Samuel Lampa wrote:

 I think I might see something of what you mean.

 For example, the highly successful (not the least in terms of 
 ecosystem) Drupal project, has many subprojects, for improving 
 things around Drupal (specific set of modules/plug-ins/project 
 infrastructure and many other things), which are organised as 
 groups on their group-discussion site (powered by the "organic 
 groups" drupal module, which is highly suited for this kind of 
 self-organizing groups).

 http://groups.drupal.org/

 (Anyone can start a new group, to facilitate community 
 self-organization)

 You mentioned the main criteria, to get something such to work. 
 which not many systems have, namely making it easy to keeping 
 track of who is interested/involved in what, in order to 
 efficiently interact with the relevant sub-group of the 
 community.

 What D has in this area, seems to be the DSource website, and 
 forums:
 http://www.dsource.org/forums/

 I don't know how it works out though (haven't participated), 
 but it turns out that the above feature is missing: It's not 
 easy to keep track of who are involved (subscribe to) in 
 projects. Also, it could maybe be made easier to find for 
 newcomers?

 (Maybe Drupal with "organic groups" could be used to set up an 
 improved group discussion site? :) )

 Cheers
 // Samuel


The groups do not seem to be hierarchical based but just a linear 
collection. Any method that just uses a linear collection is bad 
because the project can't be logically divided into smaller 
chunks in a uniform way(in a way that is uniform across the 
software and logical to the user). Essentially the drupal groups, 
as I have just looked, is a collection of projects in a list but 
each project is not a collection of sub-projects and there is no 
commonality between all the stuff I talked about before(forums, 
users/collaboration, etc...)

I am not sure about gerrit as I've just looked at it but best I 
can tell it is just a simple review tool. What I am talking about 
would have code review built in but the main thing is the 
hierarchical design which allows projects to be divide into 
sub-projects and tasks which users can dig down to what they want 
to contribute at(from the overview design of a project to a 
single line of code) but there is a logical separation as 
children can't (necessarily) see parents(although you will need 
the ability to have non-parental dependencies).

I think the most important aspects is the hierarchical nature and 
the commonality of nodes(All being project based and having the 
exact same interface for the user: project forums, collaboration 
tools, overview of child projects, etc).

The hierarchy not only allows one to quickly find any sub-project 
and project area(e.g., it categorizes projects) but also allows 
one to sort of "build" the whole D structure using scripts on 
each node(maintained by the maintainers of that node). Each node 
into the tree will push it's changes up to the parent node and 
the maintainers can accept those pushes which will chain finally 
to the root node which will be the current snapshot of D.

The easy of use comes from the tools and user interface(I'm 
thinking more of a like project tree diagram.

Test builds could be carried out by individuals relatively easily 
by "downloading" the whole D hierarchy(for arguments sake). If 
I'm trying to implement a GC for D I could write my own(or fork 
another) and then build D using that GC and it will be integrated 
in automatically(or easily, unless errors). One wouldn't need to 
compile all of D, only what changed, which would be only your 
project, so it would be quick). Of course, if all you are working 
on effects the D compiler you could terminate the build process 
earlier(for slightly quicker results and possibly smaller 
footprint on the download).
Feb 21 2013
prev sibling parent reply Jacob Carlborg <doob me.com> writes:
On 2013-02-21 13:53, js.mdnq wrote:

 This is an idea I have that I believe can help D become in time one of
 the most popular, if not the, programming language.

 The D language, excluding some of known issues, has many great language
 constructs that set it above the mainstream programming languages today.
 While there are many competitors to D, D seems to have stronger
 fundamentals and IMO a better language syntax steaming from it's
 relation to C/C++.

 There are several problems with D as it stands but the main one is the
 userbase. For proper development there must be users or are willing to
 invest the time and energy to create the tools to make more users want
 to use D(which will result in a cycle (or rather a spiral)).


Something like this, but with forms as well:

https://www.ruby-toolbox.com/

-- 
/Jacob Carlborg
Feb 21 2013
parent reply "js.mdnq" <js_adddot+mdng gmail.com> writes:
On Thursday, 21 February 2013 at 21:32:56 UTC, Jacob Carlborg 
wrote:

 On 2013-02-21 13:53, js.mdnq wrote:

 This is an idea I have that I believe can help D become in 
 time one of
 the most popular, if not the, programming language.

 The D language, excluding some of known issues, has many great 
 language
 constructs that set it above the mainstream programming 
 languages today.
 While there are many competitors to D, D seems to have stronger
 fundamentals and IMO a better language syntax steaming from 
 it's
 relation to C/C++.

 There are several problems with D as it stands but the main 
 one is the
 userbase. For proper development there must be users or are 
 willing to
 invest the time and energy to create the tools to make more 
 users want
 to use D(which will result in a cycle (or rather a spiral)).


 Something like this, but with forms as well:

 https://www.ruby-toolbox.com/


lol, someone stole my idea! ;) Looks like it has many features I 
am talkig about. I'm not sure if you can dive down an endless 
hierarchy though but seems to have a lot of the features(I 
haven't checked it out enough to know but)

(the organizational and visual theme looks a bit chaotic to me 
but seems to have docs, code, and other stuff(no forum per 
project))

I would say it has a close concept to mine but not exactly the 
same.

mine would be implemented something like

class node {
    node[] children; // child nodes
    forum(children);  // A forum interface for this node with 
access to the child forums
    management; // List of people who manage this project and all 
children
    buildscript; // Some type of script to do something
    wiki(children); // A help/info system about this node and all 
children
    dependencies;  // a list of cross-references for internode 
dependencies.
    data;  // Data for the node (source code or whatever)
    statistics;  // Statistics about the node
    info;        // node info(name, version, etc...)
    ...
}

Each element of node has a web page associated with it to 
visualize the data.

The web page for a node would be something like an html page that 
shows a forum link, wiki link, dependencies link, info about the 
project, the members, related projects(siblings), etc...

Unlike ruby, each node can contain subnodes which can deal with 
specific areas of a larger project(e.g., for an ide, just the 
gui, or some specific part of the gui).

This way large projects can be infinitely parsed down to very 
small bits that anyone can work on and which can propagate up to 
the root node(and this is the power of such a hierarchy).
Feb 21 2013
parent Jacob Carlborg <doob me.com> writes:
On 2013-02-22 01:22, js.mdnq wrote:


 lol, someone stole my idea! ;) Looks like it has many features I am
 talkig about. I'm not sure if you can dive down an endless hierarchy
 though but seems to have a lot of the features(I haven't checked it out
 enough to know but)


What's really nice about that site is that if I need a login system or, 
say, a state machine for my Rails site I can go there and find a plugin.

-- 
/Jacob Carlborg
Feb 21 2013