www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Re-thinking D's modules

reply "Dejan Lekic" <dejan.lekic gmail.com> writes:
There are several places for D module system to improve.
One thing we discussed in the past is the versioning, and as far 
as I remember, we did not come to any constructive conclusion.

Java has been criticised often for not having modules. Apparently 
Java 9 SE will have them, and in my humble opinion, Java 9 module 
system is going to be far more powerful (or perhaps better word 
would be USEFUL) than what D currently has.

More about Java Jigsaw: 
http://cr.openjdk.java.net/~mr/jigsaw/notes/jigsaw-big-picture-01

Why is this better? - Speaking from a (senior) software engineer 
point of view, Java Jigsaw is engineered for large systems where 
versioning, module-dependency, and module-restrictions are very 
important.

I do not like few things about Jigsaw, but most of the things 
they plan there simply make sense, especially the versioning and 
module-restrictions, which I urge D developers to take a look and 
come up with something similar for D2 or D3... This is extremely 
useful, and will be even more useful once we have shared 
libraries where we can have N different shared libraries that 
contain the same module, but different version of it...

Kind regards
Jul 18 2012
next sibling parent reply "Paulo Pinto" <pjmlp progtools.org> writes:
On Wednesday, 18 July 2012 at 08:08:21 UTC, Dejan Lekic wrote:

 There are several places for D module system to improve.
 One thing we discussed in the past is the versioning, and as 
 far as I remember, we did not come to any constructive 
 conclusion.

 Java has been criticised often for not having modules. 
 Apparently Java 9 SE will have them, and in my humble opinion, 
 Java 9 module system is going to be far more powerful (or 
 perhaps better word would be USEFUL) than what D currently has.

 More about Java Jigsaw: 
 http://cr.openjdk.java.net/~mr/jigsaw/notes/jigsaw-big-picture-01

 Why is this better? - Speaking from a (senior) software 
 engineer point of view, Java Jigsaw is engineered for large 
 systems where versioning, module-dependency, and 
 module-restrictions are very important.

 I do not like few things about Jigsaw, but most of the things 
 they plan there simply make sense, especially the versioning 
 and module-restrictions, which I urge D developers to take a 
 look and come up with something similar for D2 or D3... This is 
 extremely useful, and will be even more useful once we have 
 shared libraries where we can have N different shared libraries 
 that contain the same module, but different version of it...

 Kind regards


Jigsaw has just been dropped from Java 8.
http://mreinhold.org/blog/late-for-the-train

Still I would say this is so relevant that most current build 
systems
have versions as first class concept.

For those that don't know .NET, due to the DLL Hell experience, 
Microsoft
has built version support in the CLR from day 1.

--
Paulo
Jul 18 2012
next sibling parent reply Russel Winder <russel winder.org.uk> writes:
On Wed, 2012-07-18 at 11:00 +0200, Paulo Pinto wrote:
[=E2=80=A6]

 Java has been criticised often for not having modules.=20




In the beginning there was WORA which assumed so much that was
unrealistic. It's consequence has been that artefact versioning is a
complete mess in Java.  OK so the Maven repository and build frameworks
like Gradle and Maven are getting close to defeating the problem. Sadly
transitive dependencies are still a nightmare.

Of course I have probably violated a Sun/Oracle patent by even
mentioning the acronym WORA.


 Apparently Java 9 SE will have them, and in my humble opinion,=20
 Java 9 module system is going to be far more powerful (or=20
 perhaps better word would be USEFUL) than what D currently has.

 More about Java Jigsaw:=20
 http://cr.openjdk.java.net/~mr/jigsaw/notes/jigsaw-big-picture-01




Assuming that it doesn't get bounced to Java 10. It already got bounced
from Java 7 to Java 8 and now Java 8 to Java 9.


 Why is this better? - Speaking from a (senior) software=20
 engineer point of view, Java Jigsaw is engineered for large=20
 systems where versioning, module-dependency, and=20
 module-restrictions are very important.

 I do not like few things about Jigsaw, but most of the things=20
 they plan there simply make sense, especially the versioning=20
 and module-restrictions, which I urge D developers to take a=20
 look and come up with something similar for D2 or D3... This is=20
 extremely useful, and will be even more useful once we have=20
 shared libraries where we can have N different shared libraries=20
 that contain the same module, but different version of it...




Isn't the real question why is the same dynamic library linking problem
happening again. Has nothing been learned from UNIX shared objects and
Windows DLLs?

Go solves the problem by refusing all notion of dynamic linking and
insisting on static linking of all applications.


 Kind regards


=20
 Jigsaw has just been dropped from Java 8.
 http://mreinhold.org/blog/late-for-the-train


That is because to all intents and purposes Jigsaw is vapourware.  The
competition, that the Jigsaw folks are studiously ignoring, is OSGi. I
know there are some slight differences in overall aim and goal between
the two, but in the end the effect is the same: being able to run
different versions of the same artefact on the same JVM in the same
application.


 Still I would say this is so relevant that most current build=20
 systems
 have versions as first class concept.


Certainly Gradle and Maven support this idea very well.  Sadly Make,
CMake, Autotools, SCons, Waf,=E2=80=A6 tend to delegate the problem to some=
one
else.


 For those that don't know .NET, due to the DLL Hell experience,=20
 Microsoft
 has built version support in the CLR from day 1.


But, as ever, Microsoft see things like this as a way to try and get
everyone to use Windows via proprietary lock-in rather than by trying to
educate people about possible solutions. Otherwise we would already have
solved the problem rather than have the shared object mess we currently
have, OSGi, Jigsaw, the mess that is Debian and Fedora support for
multiple versions of dynamically linked libraries and Java artefacts.

Sadly I have nothing constructive to say except that which stems from
the above: in a large system comprising subsystems with transitive
dependencies, if I cannot use different versions of the same subsystem
in the overall system then everything will break. Especially in the face
of dependency injection.

--=20
Russel.
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D
Dr Russel Winder      t: +44 20 7585 2200   voip: sip:russel.winder ekiga.n=
et
41 Buckmaster Road    m: +44 7770 465 077   xmpp: russel winder.org.uk
London SW11 1EN, UK   w: www.russel.org.uk  skype: russel_winder
Jul 18 2012
next sibling parent "Paulo Pinto" <pjmlp progtools.org> writes:
On Wednesday, 18 July 2012 at 09:25:03 UTC, Russel Winder wrote:

 […]
 Isn't the real question why is the same dynamic library linking 
 problem
 happening again. Has nothing been learned from UNIX shared 
 objects and
 Windows DLLs?

 Go solves the problem by refusing all notion of dynamic linking 
 and
 insisting on static linking of all applications.


As I already mentioned a few times in Gonuts, static linking makes
it hard to implement certain types of applications.

Plus, the Go folks seem to live in open source land, where the 
code
for all 3rd party libraries is available, and no one has any 
issues
with licenses that forbade static linking.

Their attitude to keep versioning out of "go get" is a reflection 
of this.


 For those that don't know .NET, due to the DLL Hell 
 experience, Microsoft
 has built version support in the CLR from day 1.


 But, as ever, Microsoft see things like this as a way to try 
 and get
 everyone to use Windows via proprietary lock-in rather than by 
 trying to
 educate people about possible solutions.


To be fair to Microsoft, I am yet to see a commercial vendor that
does not do that.

The ones that do, usually have some consulting agenda, 
certifications,
whatever, to sell along their "free and standard" solution.


--
Paulo
Jul 18 2012
prev sibling next sibling parent reply Marco Leise <Marco.Leise gmx.de> writes:
Am Wed, 18 Jul 2012 10:24:36 +0100
schrieb Russel Winder <russel winder.org.uk>:


 Go solves the problem by refusing all notion of dynamic linking and
 insisting on static linking of all applications.


... not possible? While I learned a while back that system calls exist and =
are not dynamically linked, I wonder how Go goes about:

* basic system libraries (static linking to kernel32.dll?)
* executable bloat from large OO toolkits like Qt
* increased memory footprint by not allowing shared instances of DLLs/SOs
* modular development (e.g. separating "server" and "client" code in games)
* dynamically loadable plugins/extensions
* security and bug fixes to libraries used in dozens of programs
  (-> recompile of all library users ?)

--=20
Marco
Jul 18 2012
parent reply "Paulo Pinto" <pjmlp progtools.org> writes:
On Wednesday, 18 July 2012 at 09:59:03 UTC, Marco Leise wrote:

 Am Wed, 18 Jul 2012 10:24:36 +0100
 schrieb Russel Winder <russel winder.org.uk>:


 Go solves the problem by refusing all notion of dynamic 
 linking and
 insisting on static linking of all applications.


 ... not possible? While I learned a while back that system 
 calls exist and are not dynamically linked, I wonder how Go 
 goes about:

 * basic system libraries (static linking to kernel32.dll?)
 * executable bloat from large OO toolkits like Qt


Don't flame me, just giving the typical set of answers in gonuts,
which you can easily find when searching for .so/.dll discussions.

Use strip to reduce executable size.


 * increased memory footprint by not allowing shared instances 
 of DLLs/SOs
 * modular development (e.g. separating "server" and "client" 
 code in games)


Separate code in common packages, while generating separate 
executables.


 * dynamically loadable plugins/extensions


 From the security point of view loadable plugins are not good.

Better make use of IPC to communicate between plugins.

If you need performance, make use of shared memory as IPC 
mechanism,
which incidentally invalidates the security reasons.


 * security and bug fixes to libraries used in dozens of programs
   (-> recompile of all library users ?)


yes
Jul 18 2012
parent "Tobias Pankrath" <tobias pankrath.net> writes:
On Wednesday, 18 July 2012 at 11:21:08 UTC, Paulo Pinto wrote:


 * dynamically loadable plugins/extensions


 From the security point of view loadable plugins are not good.

 Better make use of IPC to communicate between plugins.


Since we kind of have a "never recover from error"-policy that's 
the only way to have plugins in D too, if you want to have 
plugins crash your main application.
Jul 19 2012
prev sibling next sibling parent "Dejan Lekic" <dejan.lekic gmail.com> writes:
 Go solves the problem by refusing all notion of dynamic linking 
 and
 insisting on static linking of all applications.


God, I feel sorry for programmers who use Go, in that case...
Jul 18 2012
prev sibling next sibling parent reply Wouter Verhelst <wouter grep.be> writes:
Russel Winder <russel winder.org.uk> writes:

 I do not like few things about Jigsaw, but most of the things they
 plan there simply make sense, especially the versioning and
 module-restrictions, which I urge D developers to take a look and
 come up with something similar for D2 or D3... This is extremely
 useful, and will be even more useful once we have shared libraries
 where we can have N different shared libraries that contain the
 same module, but different version of it...




 Isn't the real question why is the same dynamic library linking
 problem happening again. Has nothing been learned from UNIX shared
 objects and Windows DLLs?


ELF (aka "unix") shared objects actually do support versioned symbols
(and have since at least 1999) though they're not default and using them
makes building a shared library somewhat more complex, as you have to
tell the linker about ABI versions. But with properly done versioned
symbols, linking two different versions of the same shared library into
the same process space is perfectly safe; the C library itself uses it,
for instance:

wouter carillon:~$ objdump -T /lib/x86_64-linux-gnu/libc.so.6  | head -n 20

/lib/x86_64-linux-gnu/libc.so.6:     file format elf64-x86-64

DYNAMIC SYMBOL TABLE:
000000000001eb80 l    d  .text	0000000000000000              .text
000000000037d738 l    d  .tdata	0000000000000000              .tdata
0000000000000000      DO *UND*	0000000000000000  GLIBC_PRIVATE _rtld_global
0000000000000000      DO *UND*	0000000000000000  GLIBC_PRIVATE
__libc_enable_secure
0000000000000000      DF *UND*	0000000000000000  GLIBC_2.3   __tls_get_addr
0000000000000000      DO *UND*	0000000000000000  GLIBC_PRIVATE _rtld_global_ro
0000000000000000  w   D  *UND*	0000000000000000              _dl_starting_up
0000000000000000      DO *UND*	0000000000000000  GLIBC_PRIVATE _dl_argv
00000000000709d0 g    DF .text	0000000000000115  GLIBC_2.2.5 putwchar
000000000008bbf0 g    DF .text	000000000000001e  GLIBC_2.2.5 __strspn_c1
00000000000ec750 g    DF .text	0000000000000017  GLIBC_2.4   __gethostname_chk
000000000008bc10 g    DF .text	000000000000001a  GLIBC_2.2.5 __strspn_c2
00000000000f2660 g    DF .text	00000000000000a5  GLIBC_2.2.5 setrpcent
0000000000093760 g    DF .text	000000000000000a  GLIBC_2.2.5 __wcstod_l
000000000008bc30 g    DF .text	0000000000000022  GLIBC_2.2.5 __strspn_c3
00000000000d8c70 g    DF .text	0000000000000025  GLIBC_2.3.2 epoll_create

the GLIBC_* thingies (apart from the GLIBC_PRIVATE ones) are symbol
versions. As you can see, it's even safe to have multiple versions in
the same shared object, and (though it's not shown here) you can have
multiple implementations of a particular symbol (but with different
versions) in one shared object.

For some details on how it works, see
http://refspecs.linuxbase.org/LSB_3.1.1/LSB-Core-generic/LSB-Core-generic/symversion.html


 Go solves the problem by refusing all notion of dynamic linking and
 insisting on static linking of all applications.


"General Motors solves the problem of fatal car crashes by building cars
without engines"

That's not a solution, that's avoiding the issue.

Having said that, doing shared libraries properly is fairly complex, so
it's not necessarily a bad idea to push the problem into the future
until you've thought about it a lot; otherwise you'll probably end up
with a broken solution.


 Certainly Gradle and Maven support this idea very well.  Sadly Make,
 CMake, Autotools, SCons, Waf,… tend to delegate the problem to someone
 else.


autotools has libtool, which does the shared object stuff, and which can
do symbol versioning.

I don't know about cmake; and scons and waf are both crap, so it's not
surprising they've not even heard of symbol versioning.

Make, of course, is a programming language for writing build systems,
it's not a build system by itself.

-- 
The volume of a pizza of thickness a and radius z can be described by
the following formula:

pi zz a
Jul 20 2012
parent reply Russel Winder <russel winder.org.uk> writes:
On Sat, 2012-07-21 at 02:17 +0200, Wouter Verhelst wrote:
[...]

 I don't know about cmake; and scons and waf are both crap, so it's not
 surprising they've not even heard of symbol versioning.


I disagree, I think SCons and Waf (along with Gradle) make Make, CMake,
Autotools, etc., etc. look like "past their use-by date" tools from the
1970s. You have to justify your opinion that they are "crap", simply
saying they are does not make it so.

--=20
Russel.
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D
Dr Russel Winder      t: +44 20 7585 2200   voip: sip:russel.winder ekiga.n=
et
41 Buckmaster Road    m: +44 7770 465 077   xmpp: russel winder.org.uk
London SW11 1EN, UK   w: www.russel.org.uk  skype: russel_winder
Jul 22 2012
parent Wouter Verhelst <wouter grep.be> writes:
Russel Winder <russel winder.org.uk> writes:


 On Sat, 2012-07-21 at 02:17 +0200, Wouter Verhelst wrote:
 [...]

 I don't know about cmake; and scons and waf are both crap, so it's not
 surprising they've not even heard of symbol versioning.


 I disagree, I think SCons and Waf (along with Gradle) make Make, CMake,
 Autotools, etc., etc. look like "past their use-by date" tools from the
 1970s. You have to justify your opinion that they are "crap", simply
 saying they are does not make it so.


(this is fairly off-topic here, so I won't comment on this subthread
anymore -- please reply by private mail if you want more details)

Waf wants you to keep the entire waf source inside your distribution
tarball. This makes it extremely hard for a distribution to update all
their packages for, say, security issues in waf, and makes your
distribution tarball huge for no good reason. Additionally, waf has a
tendency to break API every other release last I checked, which makes it
difficult to upgrade. I happen to think the waf maintainer is fairly
nuts too, but that's a personal opinion. I haven't looked close enough
at waf to give you much detail about it, however.

SCons happened to be used by a package I used to maintain for Debian. It
lacks a number of fairly important features, such as the ability to
cross-build, do staged installs (i.e., something like "make DESTDIR=/tmp
install" with automake), and a number of other things. Additionally, for
the things it does support, the fact that you can just inject arbitrary
python code everywhere means that in most cases people using SCons for
their build system will (inadvertently?) break it, since they don't know
about the feature or think it doesn't matter for their project. It's
just a pain to use for people building software with it who have
nonstandard needs.

Autotools, while a bit warty in some areas, actually works. Once I read
the manual (which, granted, is something that shouldn't be necessary in
the first place) and understood how it worked, I've never wanted to use
anything else anymore.

There are many people out there who think they can do a better job than
autotools, and they all come up with a substandard YABAR[1]. I'll be the
first to agree that autotools could use *quite* some improvement in the
usability department, but replacing a body of code which has been 20
years in the making so you come up with similar functionality isn't
something you can do in two weeks of coding. Yes, granted, waf and SCons
have been worked on for a while longer than that. Still.

[1] Yet Another Bloody Autotools Replacement

-- 
The volume of a pizza of thickness a and radius z can be described by
the following formula:

pi zz a
Jul 22 2012
prev sibling next sibling parent reply "Kagamin" <spam here.lot> writes:
On Wednesday, 18 July 2012 at 09:25:03 UTC, Russel Winder wrote:

 For those that don't know .NET, due to the DLL Hell 
 experience, Microsoft
 has built version support in the CLR from day 1.


 But, as ever, Microsoft see things like this as a way to try 
 and get
 everyone to use Windows via proprietary lock-in rather than by 
 trying to
 educate people about possible solutions.


AFAIK MS tried to use java, but they wasn't allowed to do it. 
They had to invent a completely alternative solution. And .net is 
a possible solution, and MS educates people about it.

Also versioning of components is done by COM for ages, though the 
application has to be designed and written with COM in mind. 
Components don't even have to be implemented in the same language 
nor reside in the same process, nor on the same machine.
Jul 21 2012
next sibling parent Wouter Verhelst <wouter grep.be> writes:
"Kagamin" <spam here.lot> writes:


 On Wednesday, 18 July 2012 at 09:25:03 UTC, Russel Winder wrote:

 For those that don't know .NET, due to the DLL Hell experience,
 Microsoft
 has built version support in the CLR from day 1.


 But, as ever, Microsoft see things like this as a way to try and get
 everyone to use Windows via proprietary lock-in rather than by
 trying to
 educate people about possible solutions.


 AFAIK MS tried to use java, but they wasn't allowed to do it.


They were, but they had to make changes -- and that, they weren't
allowed to do.

-- 
The volume of a pizza of thickness a and radius z can be described by
the following formula:

pi zz a
Jul 21 2012
prev sibling parent Russel Winder <russel winder.org.uk> writes:
On Sat, 2012-07-21 at 11:26 +0200, Kagamin wrote:
[=E2=80=A6]

 AFAIK MS tried to use java, but they wasn't allowed to do it.=20
 They had to invent a completely alternative solution. And .net is=20
 a possible solution, and MS educates people about it.


Microsoft changed the Java Virtual Machine (JVM) on Windows and Sun
objected to them doing this. Sun's position was that the JVM should be
the same on all platforms =E2=80=93 which is an entirely reasonable positio=
n for
them to take.  Microsoft didn't think so and tried to persevere with
changing the JVM so Sun took them to court and won. Which was right.


a platform other than Windows.

But this has nothing to do with D!

--=20
Russel.
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D
Dr Russel Winder      t: +44 20 7585 2200   voip: sip:russel.winder ekiga.n=
et
41 Buckmaster Road    m: +44 7770 465 077   xmpp: russel winder.org.uk
London SW11 1EN, UK   w: www.russel.org.uk  skype: russel_winder
Jul 21 2012
prev sibling parent "Kagamin" <spam here.lot> writes:
On Wednesday, 18 July 2012 at 09:25:03 UTC, Russel Winder wrote:

 For those that don't know .NET, due to the DLL Hell 
 experience, Microsoft
 has built version support in the CLR from day 1.


 But, as ever, Microsoft see things like this as a way to try 
 and get
 everyone to use Windows via proprietary lock-in


Also why mono exists, if it's meant to be a lock-in?
Jul 21 2012
prev sibling parent "Dejan Lekic" <dejan.lekic gmail.com> writes:
 Jigsaw has just been dropped from Java 8.
 http://mreinhold.org/blog/late-for-the-train


Erm, read my original post - I did not mention Java 8, i wrote 
"Java 9 SE"...
Jul 18 2012
prev sibling next sibling parent reply Jacob Carlborg <doob me.com> writes:
On 2012-07-18 10:08, Dejan Lekic wrote:

 There are several places for D module system to improve.
 One thing we discussed in the past is the versioning, and as far as I
 remember, we did not come to any constructive conclusion.

 Java has been criticised often for not having modules. Apparently Java 9
 SE will have them, and in my humble opinion, Java 9 module system is
 going to be far more powerful (or perhaps better word would be USEFUL)
 than what D currently has.

 More about Java Jigsaw:
 http://cr.openjdk.java.net/~mr/jigsaw/notes/jigsaw-big-picture-01

 Why is this better? - Speaking from a (senior) software engineer point
 of view, Java Jigsaw is engineered for large systems where versioning,
 module-dependency, and module-restrictions are very important.

 I do not like few things about Jigsaw, but most of the things they plan
 there simply make sense, especially the versioning and
 module-restrictions, which I urge D developers to take a look and come
 up with something similar for D2 or D3... This is extremely useful, and
 will be even more useful once we have shared libraries where we can have
 N different shared libraries that contain the same module, but different
 version of it...

 Kind regards


Something like: 
https://github.com/jacob-carlborg/orbit/wiki/Orbit-Package-Manager-for-D

-- 
/Jacob Carlborg
Jul 18 2012
next sibling parent reply Nick Sabalausky <SeeWebsiteToContactMe semitwist.com> writes:
On Wed, 18 Jul 2012 11:12:22 +0200
Jacob Carlborg <doob me.com> wrote:

 
 Something like: 
 https://github.com/jacob-carlborg/orbit/wiki/Orbit-Package-Manager-for-D
 


Speaking of, how's that coming along?
Jul 18 2012
parent Jacob Carlborg <doob me.com> writes:
On 2012-07-18 12:07, Nick Sabalausky wrote:


 Speaking of, how's that coming along?


I haven't been working on that for a while. I've been working on DStep. 
The most basic functionality is there:

* Creating packages
* Installing packages
* Downloading packages
* Uploading packages

Although no real dependency algorithm is implemented yet. I think 
there's a really simple one, if I recall correctly.

https://github.com/jacob-carlborg/dstep

-- 
/Jacob Carlborg
Jul 18 2012
prev sibling parent reply "Dejan Lekic" <dejan.lekic gmail.com> writes:
 Something like: 
 https://github.com/jacob-carlborg/orbit/wiki/Orbit-Package-Manager-for-D


I am aware of the Orbit project, but what Jigsaw will do for 
Java, and what similar built-in module versioning in other 
languages do, is to give control of pieces of large software 
systems. Having module version as part of the language will give 
tools like Orbit a nice, standardized way to know what is the 
module version of a D source file it is currently processing. 
Later on, it may help compilers create SO/DLL version information 
if a certain module becomes a library...
Say you have a product, and you want to provide support for all 
legacy versions. One way to that is to put all old(er) versions 
of a module into a single library. If the module is not 
versioned, you would have to use external module information, 
typically stored in bunch of config files, and what is worse is 
quite often, if you lose these files, you have no idea what 
module version it was... (Unless you are pedantic, and write that 
information in the header comment of the module)

I use Maven a lot, every day, Jigsaw is not a replacement for 
Maven, it is a help to Maven, coming from the language itself.
Jul 18 2012
parent Jacob Carlborg <doob me.com> writes:
On 2012-07-18 12:23, Dejan Lekic wrote:


 I am aware of the Orbit project, but what Jigsaw will do for Java, and
 what similar built-in module versioning in other languages do, is to
 give control of pieces of large software systems. Having module version
 as part of the language will give tools like Orbit a nice, standardized
 way to know what is the module version of a D source file it is
 currently processing. Later on, it may help compilers create SO/DLL
 version information if a certain module becomes a library...
 Say you have a product, and you want to provide support for all legacy
 versions. One way to that is to put all old(er) versions of a module
 into a single library. If the module is not versioned, you would have to
 use external module information, typically stored in bunch of config
 files, and what is worse is quite often, if you lose these files, you
 have no idea what module version it was... (Unless you are pedantic, and
 write that information in the header comment of the module)

 I use Maven a lot, every day, Jigsaw is not a replacement for Maven, it
 is a help to Maven, coming from the language itself.


Yeah, sure. With language support we could do much more. Orbit is 
focused at working without language support, at least as a start.

-- 
/Jacob Carlborg
Jul 18 2012
prev sibling next sibling parent "bearophile" <bearophileHUGS lycos.com> writes:
Dejan Lekic:


 I do not like few things about Jigsaw, but most of the things 
 they plan there simply make sense, especially the versioning 
 and module-restrictions, which I urge D developers to take a 
 look and come up with something similar for D2 or D3...


Keep in mind that D language seems designed for its parts to be 
minimal (this means "the simplest thing that works"), and that 
the D module system is not yet fully implementing its current 
simple design.

In past I have asked for those "Entry points" (that it's an 
unsolved problem in D), and I appreciate those "Exports". The 
versioning too seems useful.

Bye,
bearophile
Jul 18 2012
prev sibling next sibling parent reply Marco Leise <Marco.Leise gmx.de> writes:
Am Wed, 18 Jul 2012 10:08:19 +0200
schrieb "Dejan Lekic" <dejan.lekic gmail.com>:


 There are several places for D module system to improve.
 One thing we discussed in the past is the versioning, and as far 
 as I remember, we did not come to any constructive conclusion.
 
 Java has been criticised often for not having modules. Apparently 
 Java 9 SE will have them, and in my humble opinion, Java 9 module 
 system is going to be far more powerful (or perhaps better word 
 would be USEFUL) than what D currently has.
 
 More about Java Jigsaw: 
 http://cr.openjdk.java.net/~mr/jigsaw/notes/jigsaw-big-picture-01
 
 Why is this better? - Speaking from a (senior) software engineer 
 point of view, Java Jigsaw is engineered for large systems where 
 versioning, module-dependency, and module-restrictions are very 
 important.
 
 I do not like few things about Jigsaw, but most of the things 
 they plan there simply make sense, especially the versioning and 
 module-restrictions, which I urge D developers to take a look and 
 come up with something similar for D2 or D3... This is extremely 
 useful, and will be even more useful once we have shared 
 libraries where we can have N different shared libraries that 
 contain the same module, but different version of it...
 
 Kind regards
 


Shared library versioning and dependency management is traditionally done at
the OS level (WinSxS, soname, package managers). That may also apply to what is
exported aka visible from a library aka module (e.g. export-all policy). Unlike

consider the target platform in addition to the module version. In addition to
this, the scope of a Java module can be anything, up to a complete tomcat web
server, whereas the scope of a D module is a single file. A Java module can
contain several packages, a D module cannot.

The main difference is in the eco system, in my opinion. Java or .NET are
platforms on their own while systems languages integrate with the hardware and
OS (starting with calling conventions, up to high-level dependency management).
Java is free to innovate here, it is 'their' platform.

Maybe you can elaborate a bit more on what module-restrictions are useful for.
I think they are the only feature that I haven't seen in package managers or
programming languages. Also the compiler doesn't currently scan shared
libraries or even require them to exist. The resolution of symbols is deferred
to the linking stage where there are existing mechanisms to hide symbols from a
library. And some people like the export-all policy with no hidden symbols, so
we are in for a hot discussion on details. :D

P.S.: I am all for a D package manager for developers, that can pull source
code and bindings from the web.

-- 
Marco
Jul 18 2012
parent reply "Paulo Pinto" <pjmlp progtools.org> writes:
On Wednesday, 18 July 2012 at 09:43:09 UTC, Marco Leise wrote:

 [..]

 languages must also consider the target platform in addition to 
 the module version. [...]



if I wished to do so.

--
Paulo
Jul 18 2012
parent reply Marco Leise <Marco.Leise gmx.de> writes:
Am Wed, 18 Jul 2012 11:56:45 +0200
schrieb "Paulo Pinto" <pjmlp progtools.org>:



 if I wished to do so.
 
 --
 Paulo


Forum.getUserByName("Paulo Pinto").addAttribute("nit-picky"); ;)

-- 
Marco
Jul 18 2012
parent "Paulo Pinto" <pjmlp progtools.org> writes:
On Wednesday, 18 July 2012 at 13:55:10 UTC, Marco Leise wrote:

 Am Wed, 18 Jul 2012 11:56:45 +0200
 schrieb "Paulo Pinto" <pjmlp progtools.org>:



 code if I wished to do so.
 
 --
 Paulo


 Forum.getUserByName("Paulo Pinto").addAttribute("nit-picky"); ;)


Yeah, I guess I deserve it. :)
Jul 18 2012
prev sibling parent reply "foobar" <foo bar.com> writes:
On Wednesday, 18 July 2012 at 08:08:21 UTC, Dejan Lekic wrote:

 There are several places for D module system to improve.
 One thing we discussed in the past is the versioning, and as 
 far as I remember, we did not come to any constructive 
 conclusion.

 Java has been criticised often for not having modules. 
 Apparently Java 9 SE will have them, and in my humble opinion, 
 Java 9 module system is going to be far more powerful (or 
 perhaps better word would be USEFUL) than what D currently has.

 More about Java Jigsaw: 
 http://cr.openjdk.java.net/~mr/jigsaw/notes/jigsaw-big-picture-01

 Why is this better? - Speaking from a (senior) software 
 engineer point of view, Java Jigsaw is engineered for large 
 systems where versioning, module-dependency, and 
 module-restrictions are very important.

 I do not like few things about Jigsaw, but most of the things 
 they plan there simply make sense, especially the versioning 
 and module-restrictions, which I urge D developers to take a 
 look and come up with something similar for D2 or D3... This is 
 extremely useful, and will be even more useful once we have 
 shared libraries where we can have N different shared libraries 
 that contain the same module, but different version of it...

 Kind regards


I'd say that this is going in the wrong direction.
I read an article a while ago that was really enlightening about 
this subject. The gist was that a module system is the wrong 
abstraction. Modules are an artifact of procedural thinking in 
that they are global. This hurts security, testability, etc.

Here's the link: bracha.org/newspeak-modules.pdf
Jul 19 2012
parent reply Jacob Carlborg <doob me.com> writes:
On 2012-07-19 11:18, foobar wrote:


 I'd say that this is going in the wrong direction.
 I read an article a while ago that was really enlightening about this
 subject. The gist was that a module system is the wrong abstraction.
 Modules are an artifact of procedural thinking in that they are global.
 This hurts security, testability, etc.

 Here's the link: bracha.org/newspeak-modules.pdf


Does it suggest a better approach?

-- 
/Jacob Carlborg
Jul 19 2012
parent "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Thu, Jul 19, 2012 at 01:34:31PM +0200, Jacob Carlborg wrote:

 On 2012-07-19 11:18, foobar wrote:
 

I'd say that this is going in the wrong direction.
I read an article a while ago that was really enlightening about this
subject. The gist was that a module system is the wrong abstraction.
Modules are an artifact of procedural thinking in that they are global.
This hurts security, testability, etc.

Here's the link: bracha.org/newspeak-modules.pdf


 
 Does it suggest a better approach?


[...]

I skimmed the paper briefly. Correct me if I'm misreading it, but the
approach it proposes is based on identifying classes with modules, and
requires that class names be dynamically bound; in particular,
superclasses are dynamically bound.  I don't think this will work in D's
framework.


T

-- 
Дерево держится корнями, а человек -
друзьями.
Jul 19 2012