www.digitalmars.com         C & C++   DMDScript  

D - STL as part of the D language

reply FlyTox <rox271 hotmail.com> writes:
I just discovered the D language a couple of days ago and it looks exciting.

Having programmed for quite a long time on C++, there is one thing that 
I really like and would really appreciate to see included in a new 
language : STLs. Today I cannot think without STLs, though the syntax is 
terrible. But the benefit is huge. I can see that D is digging into 
dynamic arrays and even maps. This still remains a bit basic compared 
with STL. Any chance to have more collections (sets, generic maps, 
queues etc...), iterators, comparison operators or the idea is to keep 
STL in a external template lib?
Apr 25 2004
next sibling parent reply Stephan Wienczny <wienczny web.de> writes:
FlyTox wrote:

 I just discovered the D language a couple of days ago and it looks 
 exciting.
 
 Having programmed for quite a long time on C++, there is one thing that 
 I really like and would really appreciate to see included in a new 
 language : STLs. Today I cannot think without STLs, though the syntax is 
 terrible. But the benefit is huge. I can see that D is digging into 
 dynamic arrays and even maps. This still remains a bit basic compared 
 with STL. Any chance to have more collections (sets, generic maps, 
 queues etc...), iterators, comparison operators or the idea is to keep 
 STL in a external template lib?
 


Matthew is writing a DTL (D Template Library). I think he will do a good 
job as he is experienced in writing template stuff for Stlsoft.
DTL will be available in one of the next dmd releases.

Stephan
Apr 25 2004
parent reply "Matthew" <matthew.hat stlsoft.dot.org> writes:
"Stephan Wienczny" <wienczny web.de> wrote in message
news:c6g3m0$18je$1 digitaldaemon.com...

 FlyTox wrote:

 I just discovered the D language a couple of days ago and it looks
 exciting.

 Having programmed for quite a long time on C++, there is one thing that
 I really like and would really appreciate to see included in a new
 language : STLs. Today I cannot think without STLs, though the syntax is
 terrible. But the benefit is huge. I can see that D is digging into
 dynamic arrays and even maps. This still remains a bit basic compared
 with STL. Any chance to have more collections (sets, generic maps,
 queues etc...), iterators, comparison operators or the idea is to keep
 STL in a external template lib?


 Matthew is writing a DTL (D Template Library). I think he will do a good
 job as he is experienced in writing template stuff for Stlsoft.
 DTL will be available in one of the next dmd releases.


Thanks. Now I've got to live up to your expectation! :)
Apr 25 2004
parent reply Ant <duitoolkit yahoo.ca> writes:
On Sun, 25 Apr 2004 23:38:46 +1000, Matthew wrote:


 
 
 Thanks. Now I've got to live up to your expectation! :)


I don't get why this is not an open project.

can you explain?
My imagination is imagining the worst possibilities.

Thanks.

Ant
Apr 25 2004
parent "Matthew" <matthew.hat stlsoft.dot.org> writes:
"Ant" <duitoolkit yahoo.ca> wrote in message
news:pan.2004.04.25.15.57.58.568212 yahoo.ca...

 On Sun, 25 Apr 2004 23:38:46 +1000, Matthew wrote:


 Thanks. Now I've got to live up to your expectation! :)


 I don't get why this is not an open project.

 can you explain?


Because I chose to do it this way. When a "DTL" was mooted several months ago,
there was no clear idea as to how a container/algorithm library would work in D;
the only attempts either used "classic" (i.e. pre-STL) style containers, or aped
STL directly. The idea floated at the time was that people were going to work on
different mechanisms, and we would eventually compare the different
implementations, and take the best ideas from them all.

I've been working on one that, rather than prescribes a single enumeration
model,
supports a diversity of them with maximal efficiency. AFAIK, no-one else has
worked on a DTL.

I've kept people up to speed with what's intended, but there can hardly be much
active collaboration, because:
    (i) until about two days ago it was still in a state of massive flux
conceptually
    (ii) the compiler does not yet support the majority of the intended
implementation, so how could we collaborate sensibly.


 My imagination is imagining the worst possibilities.


Thanks!

You're *more* than welcome to go with the initial plan, and write something
yourself, and we can compare them when they're both ready and extract the best.
Apr 25 2004
prev sibling next sibling parent reply J Anderson <REMOVEanderson badmama.com.au> writes:
FlyTox wrote:


 I just discovered the D language a couple of days ago and it looks 
 exciting.

 Having programmed for quite a long time on C++, there is one thing 
 that I really like and would really appreciate to see included in a 
 new language : STLs. Today I cannot think without STLs, though the 
 syntax is terrible. But the benefit is huge. I can see that D is 
 digging into dynamic arrays and even maps. This still remains a bit 
 basic compared with STL. Any chance to have more collections (sets, 
 generic maps, queues etc...), iterators, comparison operators or the 
 idea is to keep STL in a external template lib?



DTL is in early development by Matthew.  We should all be able to 
contribute soon.

-- 
-Anderson: http://badmama.com.au/~anderson/
Apr 25 2004
parent "Matthew" <matthew.hat stlsoft.dot.org> writes:
"J Anderson" <REMOVEanderson badmama.com.au> wrote in message
news:c6g3ra$18h9$2 digitaldaemon.com...

 FlyTox wrote:


 I just discovered the D language a couple of days ago and it looks
 exciting.

 Having programmed for quite a long time on C++, there is one thing
 that I really like and would really appreciate to see included in a
 new language : STLs. Today I cannot think without STLs, though the
 syntax is terrible. But the benefit is huge. I can see that D is
 digging into dynamic arrays and even maps. This still remains a bit
 basic compared with STL. Any chance to have more collections (sets,
 generic maps, queues etc...), iterators, comparison operators or the
 idea is to keep STL in a external template lib?



 DTL is in early development by Matthew.  We should all be able to
 contribute soon.


Very soon, I hope. Just waiting on some serious language/compiler enhancements
by
our inexplicably silent <G> leader. (Sorry Walter, but you *did* say you wanted
D
to knock the pants of everything, and C++ is very good at templates ...)
Apr 25 2004
prev sibling next sibling parent reply FlyTox <rox271 hotmail.com> writes:
Thanks for the information. So this means that DTL will be an external 
lib. Actually I was wondering why not have STL as part of the 
programming language. Who is nowaday doing any serious OOP without 
advanced object collections?

Also most COM libraries have Variant interface arrays with iterators and 
since D is offering an easy way to implement COM interfaces that would 
be nice to have collections as part of the language. Will DTL allow an 
easy COM collection implementation in order to interface with VB apps?

Should we have collections in an external lib? I believe that the next 
generation of programming language cannot ignore collections which are a 
major aspect of the nowaday OOP.
Apr 25 2004
next sibling parent J Anderson <REMOVEanderson badmama.com.au> writes:
FlyTox wrote:


 Thanks for the information. So this means that DTL will be an external 
 lib.


DTL will come with D like STL comes with C++.


-- 
-Anderson: http://badmama.com.au/~anderson/
Apr 25 2004
prev sibling next sibling parent reply Norbert Nemec <Norbert.Nemec gmx.de> writes:
What is wrong with a standard library? Of course the DTL or something like
it should be standardized and included into Phobos. (I have little doubt,
it will.)

The language itself should rather stay as slim as possible, just giving the
flexibility and power to define everything in the library.

That is just the core concept of most modern programming languages: to have
the language spec slim and general to allow powerful libraries with high
performance and clean syntax.

FlyTox wrote:


 Thanks for the information. So this means that DTL will be an external
 lib. Actually I was wondering why not have STL as part of the
 programming language. Who is nowaday doing any serious OOP without
 advanced object collections?
 
 Also most COM libraries have Variant interface arrays with iterators and
 since D is offering an easy way to implement COM interfaces that would
 be nice to have collections as part of the language. Will DTL allow an
 easy COM collection implementation in order to interface with VB apps?
 
 Should we have collections in an external lib? I believe that the next
 generation of programming language cannot ignore collections which are a
 major aspect of the nowaday OOP.
Apr 25 2004
next sibling parent "Matthew" <matthew.hat stlsoft.dot.org> writes:
 That is just the core concept of most modern programming languages: to have
 the language spec slim and general to allow powerful libraries with high
 performance and clean syntax.


Exactly right!

(Now, why couldn't I say that as simply ... <g>)


 FlyTox wrote:


 Thanks for the information. So this means that DTL will be an external
 lib. Actually I was wondering why not have STL as part of the
 programming language. Who is nowaday doing any serious OOP without
 advanced object collections?

 Also most COM libraries have Variant interface arrays with iterators and
 since D is offering an easy way to implement COM interfaces that would
 be nice to have collections as part of the language. Will DTL allow an
 easy COM collection implementation in order to interface with VB apps?

 Should we have collections in an external lib? I believe that the next
 generation of programming language cannot ignore collections which are a
 major aspect of the nowaday OOP.
Apr 25 2004
prev sibling parent reply J Anderson <REMOVEanderson badmama.com.au> writes:
Norbert Nemec wrote:


What is wrong with a standard library? 


D will probably be able to do a better job.  Also D has many features 
that overlap STL.  We want people to use the D way, not STL's.

-- 
-Anderson: http://badmama.com.au/~anderson/
Apr 25 2004
parent Norbert Nemec <Norbert.Nemec gmx.de> writes:
J Anderson wrote:


 Norbert Nemec wrote:
 

What is wrong with a standard library?


 D will probably be able to do a better job.  Also D has many features
 that overlap STL.  We want people to use the D way, not STL's.


I never talked of the C++ STL - I talked of *a* standard library. Of course,
ideas will be picked from the C++ STL, but we can certainly do a better job
that *that* in D. And how it is called in the end - who knows. Probably
just "part of the D library". (If it really is standard, it does not need a
name claiming to be standard...)
Apr 25 2004
prev sibling next sibling parent reply "Matthew" <matthew.hat stlsoft.dot.org> writes:
"FlyTox" <rox271 hotmail.com> wrote in message
news:c6g7lv$1eqm$1 digitaldaemon.com...

 Thanks for the information. So this means that DTL will be an external
 lib.


Yes.


 Actually I was wondering why not have STL as part of the
 programming language.


Two reasons.

First, language vs library is a grey area, and we choose library in this case.

Second, it is way too complex to have it all in the language.

Third, it is designed to be extensible just as STL was (but rarely is; ahem ...
http://stlsoft.org/), which leaves me, at least, guessing how it could be a
language thing.


 Who is nowaday doing any serious OOP without
 advanced object collections?


<opinionated rant>Who is doing any serious OOP? It's a dead duck!</opinionated
rant>

Who is doing any serious programming these days without serialisation (saving /
loading), serialisation (synchronisation), XML, IO, etc. etc. etc. Are all these
things to be forced into the language? Obviously not.

And in answer to your question, I know of at least one really smart chap who
rarely, if ever, uses STL.

_Even_ me, a big big fan of STL, eschews it often.

It's all horses for courses, in my opinion. In a slightly unrelated note, Walter
and I have a continuing debate over my propensity for providing free functions
along with class(es) when writing D libraries. He believes there should be only
the class-based approach, but I see D as a broad church, which should welcome
programmers with as diverse skills as is practicable. For example, we're just
arguing over the fine details of my ExeModule module, which provides an
ExeModule
class that throws exceptions, and ExeModule_Load(), ExeModule_AddRef(),
ExeModule_GetSymbol(), etc functions, which afford a more C like approach.

In the same vein, I intend to support many different enumeration models for DTL,
so long as they do not cause a sacrifice in efficiency, ease-of-use, robustness
or (within reason) maintainability.


 Also most COM libraries have Variant interface arrays with iterators and
 since D is offering an easy way to implement COM interfaces that would
 be nice to have collections as part of the language. Will DTL allow an
 easy COM collection implementation in order to interface with VB apps?


It will, via extensions, in just the same way that COMSTL, UNIXSTL, WinSTL,
InetSTL (just alpha'd on the STLSoft newsgroup today;
news://news.digitalmars.com/c++.stlsoft) all provide STL-like functionality over
disparate technology-specific and operating system APIs.


 Should we have collections in an external lib? I believe that the next
 generation of programming language cannot ignore collections which are a
 major aspect of the nowaday OOP.


Yes, for the practical reasons I've given above.
Apr 25 2004
parent Ant <duitoolkit yahoo.ca> writes:
On Sun, 25 Apr 2004 23:48:51 +1000, Matthew wrote:


 
 It's all horses for courses, in my opinion. In a slightly unrelated note,
Walter
 and I have a continuing debate over my propensity for providing free functions
 along with class(es) when writing D libraries. He believes there should be only
 the class-based approach, but I see D as a broad church, which should welcome
 programmers with as diverse skills as is practicable.


Walter is right.

Ant
Apr 25 2004
prev sibling parent Ilya Minkov <minkov cs.tum.edu> writes:
FlyTox schrieb:


 Should we have collections in an external lib? I believe that the next 
 generation of programming language cannot ignore collections which are a 
 major aspect of the nowaday OOP.


Very true. But nontheless: you cannot put everything into the core 
language, something will be still missing. Libraries are a sort of 
extension mecanism, which can fill this gap. So why not be everything a 
library? Well, it works in some languages, but with a C descentant this 
causes a bit more problems. Besides, putting everything into a compiler 
is hard and probably somewhat error-prone. So a balance has to be found. 
The core language in D already gives a lot compared to most languages 
(linear and associative container - the ones needed most often), and 
gives a solid basis for everything else... I think it should stay this 
way so far.

-eye
Apr 25 2004
prev sibling parent reply "Matthew" <matthew.hat stlsoft.dot.org> writes:
"FlyTox" <rox271 hotmail.com> wrote in message
news:c6g241$16c1$1 digitaldaemon.com...

 I just discovered the D language a couple of days ago and it looks exciting.

 Having programmed for quite a long time on C++, there is one thing that
 I really like and would really appreciate to see included in a new
 language : STLs. Today I cannot think without STLs, though the syntax is
 terrible. But the benefit is huge. I can see that D is digging into
 dynamic arrays and even maps. This still remains a bit basic compared
 with STL. Any chance to have more collections (sets, generic maps,
 queues etc...), iterators, comparison operators or the idea is to keep
 STL in a external template lib?


Here's the basic idea. DTL collections will support compile-time polymorphism by
default, and runtime polymorphism as a parameterisable option. To my knowledge
this has not been successfully done in C++ - although I have the beginnings of a
VirtualSTL (another STLSoft thing) which I'll mature throughout this year (it's
going to feature in my next book on STL) - and will represent a massive boon to
the D user community, because fans of both the STL and the .NET/Java's
interface-based approaches will be equally served.

The enumeration models supported are (or will be, just as soon as poor Walter's
satisfied my every whim; whip crack away!):



All DTL containers will present opApply() and will be freachable



All DTL sequence containers - others are undecided at present - will present an
iterator based representation to their managed sequence, in the form of begin()
and end(). Because of differences in D's support for operator overloading,
however, (see point B. below) there are differences in how iterators must be
used. But this will be syntactic only.

FYI, the current iterator for the List template, ListIterator, presents the
following interface:

    void next();
    value_type value(); // Can be accessed as property, as in  "X x = it.value;"
    int opEquals();

Since *all* DTL containers will have iterators that have the same interface,
there's no confusion with array (op*) syntax - the point simply does not exist.
Circular argument, to be sure, but it works nonetheless.



This is a concept being developed by myself and John Torjo, and features in my
soon-to-be-published book "Imperfect C++", as well as featuring even more in the
next book on STL. We're writing a pair of articles on the subject for CUJ later
in the year. At present, Ranges exists in a small STLSoft library (not yet
released) and a reasonably complete Boost library (including filters, John's
speciality).


will use Ranges to present freachable results from methods similar to the Ruby
Enumerable module (see http://ruby-lang.org; download the Win32 version, as
there's a free electronic copy of The Pragmatic Programmers book on Ruby!).


default,
by a Ranges mixin, and Walter is currently working out whether my proposed mixin
protocol - which is dirt simple, just pop the things in the mixing class unless
that class already has one of the exact signature - is feasible, and whether he
can do it before he has a stress episode. ;)


following methods (some of which may not be instantiable, of course, based on
container value type):

    template(F)
    range_type collect(F f);

    boolean contains(value_type comperand);

    template(F)
    boolean detect(F f);

    template(F)
    boolean detect(F f, out value_type value);

    value_type[] entries();


    value_type  min();

    template(F)
    value_type  min(F f);

    value_type  max();

    template(F)
    value_type  max(F f);

    template(F)
    range_type  reject(F f);

    template(F)
    range_type  select(F f);

    range_type  sort();

    template(F)
    range_type  sort(F f);







template parameters (already added for next release), mixins (to be determined)
and specialisation (extant), you will be able to parameterise your container
template on an interface (for which you've provided the appropriate mixins /
adaptors; that bit's still unclear, as I've not been able to play with actual
compiled code yet) and your container will, in addition to all its normal
behaviour, derive from and fully implement (semantically, I mean) that
interface.

This means that (i) you can live in Java/.NET Object-based collection world, if
you wish, and (ii) you can write write-once non-template functions for a given
interface, and use it with arbitrary DTL container types which you have
parameterised to your interface.


That all sounds good - at least I hope it does - but there are some restrictions
imposed by D on us, so there will be some changes to the way people use
collections, especially to those (of us) who love STL:

A. There is no implicit template instantiation. This means that the use of free
function algorithms is not going to be useful. Most of the things anyone would


cases.
In such cases, explicit instantiation will have to suffice. But given the fact
that *all* DTL collections will have a standard, sensible and mandatory set of
member types, and because D has a typeof operator, I think the pain will be
slight, if it's even noticeable.

B. There is no user-definable operator *() - I called it opDeref when I pleaded
in vain with Walter for it. (Actually, I am being dramatic for the crowd, my
plea
was a half-enthused "do we have it? Is it going to exist?" while keeping my
fingers crossed that he'd say no, which he did. With the restrictions imposed by
the _lack_ of op*(), we will have less lee-way to pansy around C compatibility
(arrays) and all collection manipulation will be much simpler and consistent.)

The ramifications of this is that there will be no compatibility in user-defined
collections with built-in arrays access by pointer. This is inline with with D
preference for array slices over pointers. Once you give in/up and accept this,
the world becomes altogether simpler. There will be a high degree of
compatibility between DTL collections and built-in arrays, in the form of



Hope that explains the current thrust of the development

Cheers


-- 
Matthew Wilson

Contributing editor, C/C++ Users Journal
    (www.synesis.com.au/articles.html#columns)
STLSoft moderator
    (http://www.stlsoft.org)

"I can't sleep nights till I found out who hurled what ball through what
apparatus" -- Dr Niles Crane

-------------------------------------------------------------------------------
Apr 25 2004
next sibling parent reply J Anderson <REMOVEanderson badmama.com.au> writes:
Matthew wrote:
<snip>

Thanks for that update Matthew.  I can't wait <g>/

-- 
-Anderson: http://badmama.com.au/~anderson/
Apr 25 2004
parent reply "Matthew" <matthew.hat stlsoft.dot.org> writes:
Oh no, not more pressure!!

<G>

"J Anderson" <REMOVEanderson badmama.com.au> wrote in message
news:c6gksu$24po$1 digitaldaemon.com...

 Matthew wrote:
 <snip>

 Thanks for that update Matthew.  I can't wait <g>/

 -- 
 -Anderson: http://badmama.com.au/~anderson/
Apr 25 2004
next sibling parent reply Stephan Wienczny <wienczny web.de> writes:
What is the current state of DTL?

How finished are DTL and the compiler changes?

Matthew wrote:

 Oh no, not more pressure!!
 
 <G>
 
 "J Anderson" <REMOVEanderson badmama.com.au> wrote in message
 news:c6gksu$24po$1 digitaldaemon.com...
 

Matthew wrote:
<snip>

Thanks for that update Matthew.  I can't wait <g>/

-- 
-Anderson: http://badmama.com.au/~anderson/
Apr 25 2004
parent reply "Matthew" <matthew.hat stlsoft.dot.org> writes:
 What is the current state of DTL?




others will pitch in on this, as soon as I can get it out there.)



 Containers - Vector, List, Set, Queue, Stack, Map - all in various stages of
Enum#N completeness. Once it's done for one, however, it'll be done for all
within a day
 Ranges - IntRange


 How finished are DTL and the compiler changes?


AFAIK, Compiler changes:

 Default template parameters - done
 Member templates - Walter's on this now
 Mixins - Waiting

 DTL completeness:

 As above


 Matthew wrote:

 Oh no, not more pressure!!

 <G>

 "J Anderson" <REMOVEanderson badmama.com.au> wrote in message
 news:c6gksu$24po$1 digitaldaemon.com...


Matthew wrote:
<snip>

Thanks for that update Matthew.  I can't wait <g>/

-- 
-Anderson: http://badmama.com.au/~anderson/
Apr 25 2004
parent Stephan Wienczny <wienczny web.de> writes:
Good to hear that...

Matthew wrote:

What is the current state of DTL?


 
 


Foreach




Iterators


hoping
 others will pitch in on this, as soon as I can get it out there.)


Ranges




interface


 
  Containers - Vector, List, Set, Queue, Stack, Map - all in various stages of
 Enum#N completeness. Once it's done for one, however, it'll be done for all
 within a day
  Ranges - IntRange
 
 

How finished are DTL and the compiler changes?


 
 
 AFAIK, Compiler changes:
 
  Default template parameters - done
  Member templates - Walter's on this now
  Mixins - Waiting
 
  DTL completeness:
 
  As above
Apr 25 2004
prev sibling parent J Anderson <REMOVEanderson badmama.com.au> writes:
Matthew wrote:


Oh no, not more pressure!!

<G>
  


I'm glad we've got someone who knows what he's doing, working on DTL.  
Thanks for the effort Matthew.

-- 
-Anderson: http://badmama.com.au/~anderson/
Apr 25 2004
prev sibling parent reply Bastiaan Veelo <Bastiaan.N.Veelo ntnu.no> writes:
Matthew wrote:

(snipped a bit)

 The enumeration models supported are (or will be, just as soon as poor Walter's
 satisfied my every whim; whip crack away!):
 

  

 

 



Apropos enumeration models, are you familiar with circulators?

http://www.cgal.org/Manual/doc_html/support_lib/Handles_and_Circs/Chapter_main.html#Section_2
http://www.cgal.org/Manual/doc_html/use_of_stl/Chapter_Circulator_stl.html

I think it would be nice to have support for circular data structures in 
the standard library. They have their uses in geometry.

Regards,
Bastiaan.
Apr 25 2004
parent reply "Matthew" <matthew.hat stlsoft.dot.org> writes:
"Bastiaan Veelo" <Bastiaan.N.Veelo ntnu.no> wrote in message
news:c6gq4n$2dge$1 digitaldaemon.com...

 Matthew wrote:

 (snipped a bit)

 The enumeration models supported are (or will be, just as soon as poor




Walter's

 satisfied my every whim; whip crack away!):










 Apropos enumeration models, are you familiar with circulators?


http://www.cgal.org/Manual/doc_html/support_lib/Handles_and_Circs/Chapter_main.html#Section_2

 http://www.cgal.org/Manual/doc_html/use_of_stl/Chapter_Circulator_stl.html

 I think it would be nice to have support for circular data structures in
 the standard library. They have their uses in geometry.


I'd certainly be interested in seeing how these might fit it. I would hope
you'll
volunteer some MDTL extension time when the compiler's ready and I've released
0.1. ;)
Apr 25 2004
parent Bastiaan Veelo <Bastiaan.N.Veelo ntnu.no> writes:
Matthew wrote:

 
 http://www.cgal.org/Manual/doc_html/support_lib/Handles_and_Circs/Chapter_main.html#Section_2
 

http://www.cgal.org/Manual/doc_html/use_of_stl/Chapter_Circulator_stl.html

I think it would be nice to have support for circular data structures in
the standard library. They have their uses in geometry.


 
 
 I'd certainly be interested in seeing how these might fit it. I would hope
you'll
 volunteer some MDTL extension time when the compiler's ready and I've released
 0.1. ;)
 
 


I can see if I am capable of that, but it will have to wait until the 
fall. I have to deliver a thesis first --- me hanging out in this NG is 
like doing the ioning... ;)

Bastiaan.
Apr 26 2004