www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - DUB colored output proposal/showcase

reply gdelazzari <giacky98.mail gmail.com> writes:
Hello everyone, I'm a new user of the language (I've been playing 
around with it for some months) and I'm really liking it.

One minor thing that I didn't particularly like was what (and 
how) DUB spits out stuff to the terminal, in the sense that I 
find the output a bit difficult to navigate i.e. I can't easily 
understand, at a first glance, what's going on. Given that I 
tried various languages recently, including their package 
manager, and seeing how much clearer their output was, I think 
DUB could also be improved in the same way. The main thing that 
would make the output clearer would be to color the output and 
format it a bit better, IMHO. It would also look way prettier.

Given that this seemed like an easy task, I already implemented a 
proof-of-concept just to see how that would look like and, in my 
opinion, now it's really clear what's going on. Take a look at 
these screenshots:

https://imgur.com/a/3RJyd6m

Some of you will have already noticed... Yes, I got really 
inspired by Rust's "Cargo" package manager, but not because I'm a 
Rust fanboy or something like that (I don't even particularly 
like it as a language), but just because of all the package 
managers I tried, Rust's one has been the simplest, cleanest, 
approachable and immediate I've ever seen. The way Cargo 
organizes and colors the stuff it prints seemed like a really 
good approach, so I thought implementing a similar thing (with 
some changes/twists) would be a good starting point. Of course 
I'm totally open to suggestions and I don't even want Dub to look 
exactly like Cargo, at all. However the concept of a colored 
"tag" at the beginning of the line looks really cool and useful 
in my opinion. All the green tags are related to the building 
process, so when your eyes see green you know it's 
compiling/linking, while package fetching/upgrading/etc... stuff 
has a yellow tag, so you immediately know Dub is doing package 
related things just by seeing yellow text. There are other minor 
changes, like removing the three dots "..." after every line 
(which, IMHO, just clutter the output) and other things.

As I said, this is just a proposal, and I will happily take any 
feedback, critiques, enhancements, etc... I also hope this won't 
offend anyone, both because I want to change Dub and because I 
got inspired by the package manger of a competitor language.

I started this thread to have a discussion on this before 
submitting any pull request (which, in the case of this change 
being apprecciated, I'll happily take the time to make).

Thanks to anyone in advance,
Giacomo
Jun 08 2018
next sibling parent reply Steven Schveighoffer <schveiguy yahoo.com> writes:
On 6/8/18 9:35 AM, gdelazzari wrote:
 Hello everyone, I'm a new user of the language (I've been playing around 
 with it for some months) and I'm really liking it.
 
 One minor thing that I didn't particularly like was what (and how) DUB 
 spits out stuff to the terminal, in the sense that I find the output a 
 bit difficult to navigate i.e. I can't easily understand, at a first 
 glance, what's going on. Given that I tried various languages recently, 
 including their package manager, and seeing how much clearer their 
 output was, I think DUB could also be improved in the same way. The main 
 thing that would make the output clearer would be to color the output 
 and format it a bit better, IMHO. It would also look way prettier.
 
 Given that this seemed like an easy task, I already implemented a 
 proof-of-concept just to see how that would look like and, in my 
 opinion, now it's really clear what's going on. Take a look at these 
 screenshots:
 
 https://imgur.com/a/3RJyd6m
Looks excellent! Two thumbs up from me. Is it cross-platform? Note on some platforms (ahem, Macos) the background is white, so this should be correctly colored for that possibility. -Steve
Jun 08 2018
next sibling parent reply gdelazzari <giacky98.mail gmail.com> writes:
On Friday, 8 June 2018 at 13:38:59 UTC, Steven Schveighoffer 
wrote:
 Looks excellent! Two thumbs up from me. Is it cross-platform?

 Note on some platforms (ahem, Macos) the background is white, 
 so this should be correctly colored for that possibility.

 -Steve
At the moment it's "probably" Linux-only, but that's because I only wanted a proof of concept and I worked on it on my Linux installation. I imported this library/Dub package https://github.com/yamadapc/d-colorize and just used it. Which, by the way, it's no-good at the moment since I saw that Dub doesn't use Dub packages itself - probably because, otherwise, you don't have a way to easily compile it without Dub itself, I guess :P so I'll need to either write my custom color outputting code within Dub's source or just import that library. Of course making it cross-platform is a mandatory thing to me. Windows also needs some specific stuff to output colors, as you can see in the library I linked, so there are definitely some things to do to support all the platforms. I may even take a look at how DMD itself outputs colored output, I guess it will be nice to keeps things consistent. As for MacOS having a different background... I don't really own a Mac nor I have ever used one before, so I don't even know how tools usually output their colored text on it. At the moment it just sets the foreground color to green/yellow/blue/whatever, without changing the background, if that was your concern. If you meant that yellow-on-white is not readable... well... I guess so. Maybe two different color palettes should be used? IDK, as I said I never used a Mac before so I don't really know how other tools handle this, maybe if some Mac user could help on this, it would be great. Thanks for the appreciation by the way!
Jun 08 2018
next sibling parent Steven Schveighoffer <schveiguy yahoo.com> writes:
On 6/8/18 9:51 AM, gdelazzari wrote:
 On Friday, 8 June 2018 at 13:38:59 UTC, Steven Schveighoffer wrote:
 Looks excellent! Two thumbs up from me. Is it cross-platform?

 Note on some platforms (ahem, Macos) the background is white, so this 
 should be correctly colored for that possibility.
At the moment it's "probably" Linux-only, but that's because I only wanted a proof of concept and I worked on it on my Linux installation. I imported this library/Dub package https://github.com/yamadapc/d-colorize and just used it. Which, by the way, it's no-good at the moment since I saw that Dub doesn't use Dub packages itself - probably because, otherwise, you don't have a way to easily compile it without Dub itself, I guess :P so I'll need to either write my custom color outputting code within Dub's source or just import that library.
Yeah, I would expect that the colorization is simply a matter of outputting the right control characters. You probably just need to include some simple stuff inside dub source itself. But I'm far from experienced on this.
 
 Of course making it cross-platform is a mandatory thing to me. Windows 
 also needs some specific stuff to output colors, as you can see in the 
 library I linked, so there are definitely some things to do to support 
 all the platforms. I may even take a look at how DMD itself outputs 
 colored output, I guess it will be nice to keeps things consistent.
 
 As for MacOS having a different background... I don't really own a Mac 
 nor I have ever used one before, so I don't even know how tools usually 
 output their colored text on it. 
I'm assuming it's similar to Linux, it's just that the background is white instead of black.
At the moment it just sets the 
 foreground color to green/yellow/blue/whatever, without changing the 
 background, if that was your concern. If you meant that yellow-on-white 
 is not readable... well... I guess so.
Yes. In fact, I've used the new vibe.d and it appears not to adjust its colorization to my screen, it's light grey on white (almost impossible to read).
 Maybe two different color 
 palettes should be used? IDK, as I said I never used a Mac before so I 
 don't really know how other tools handle this, maybe if some Mac user 
 could help on this, it would be great.
The way I would solve it is to have a "light" mode and a "dark" mode, and version the default mode based on the OS (Linux, windows, etc. all dark mode by default, macos light mode by default).
 
 Thanks for the appreciation by the way!
Thanks for the effort! -Steve
Jun 08 2018
prev sibling parent reply Uknown <sireeshkodali1 gmail.com> writes:
On Friday, 8 June 2018 at 13:51:05 UTC, gdelazzari wrote:
 On Friday, 8 June 2018 at 13:38:59 UTC, Steven Schveighoffer 
 wrote:
 Looks excellent! Two thumbs up from me. Is it cross-platform?

 Note on some platforms (ahem, Macos) the background is white, 
 so this should be correctly colored for that possibility.

 -Steve
At the moment it's "probably" Linux-only, but that's because I only wanted a proof of concept and I worked on it on my Linux installation. I imported this library/Dub package https://github.com/yamadapc/d-colorize and just used it. Which, by the way, it's no-good at the moment since I saw that Dub doesn't use Dub packages itself - probably because, otherwise, you don't have a way to easily compile it without Dub itself, I guess :P so I'll need to either write my custom color outputting code within Dub's source or just import that library. Of course making it cross-platform is a mandatory thing to me. Windows also needs some specific stuff to output colors, as you can see in the library I linked, so there are definitely some things to do to support all the platforms. I may even take a look at how DMD itself outputs colored output, I guess it will be nice to keeps things consistent. As for MacOS having a different background... I don't really own a Mac nor I have ever used one before, so I don't even know how tools usually output their colored text on it. At the moment it just sets the foreground color to green/yellow/blue/whatever, without changing the background, if that was your concern. If you meant that yellow-on-white is not readable... well... I guess so. Maybe two different color palettes should be used? IDK, as I said I never used a Mac before so I don't really know how other tools handle this, maybe if some Mac user could help on this, it would be great. Thanks for the appreciation by the way!
I love it! I have very little experience with terminal colours, but as far as colourizing text on POSIX its fairly easy. You just need to emit the right ANSI escape sequences [0]. This is what the colorize-d library does.. For Windows before Windows 10, things are more messy. You need to use `handle`s, to get the current state and then correctly set the colours. The real hard part here is adjusting the colour scheme based on the terminal background colour. [0]: https://en.wikipedia.org/wiki/ANSI_escape_code#Colors
Jun 08 2018
parent gdelazzari <giacky98.mail gmail.com> writes:
On Friday, 8 June 2018 at 15:34:06 UTC, Uknown wrote:
 I love it! I have very little experience with terminal colours, 
 but as far as colourizing text on POSIX its fairly easy. You 
 just need to emit the right ANSI escape sequences [0]. This is 
 what the colorize-d library does.. For Windows before Windows 
 10, things are more messy. You need to use `handle`s, to get 
 the current state and then correctly set the colours. The real 
 hard part here is adjusting the colour scheme based on the 
 terminal background colour.

 [0]: https://en.wikipedia.org/wiki/ANSI_escape_code#Colors
Sure, I sort of know how that works, I did some stuff some time ago. Also, regarding Windows, the colorize-d library already handles what you described. I think I'll simply take the source of colorize-d and copy-paste the files in the Dub source, as I saw they've done with some parts of Vibe.d in order to avoid having to fetch Dub packages while compiling Dub itself.
 I would love to help you on this. Is there anything in 
 particular you need help with?
Actually, it's not a difficult task. But, being a bit of a perfectionst, I already spotted the possibility of writing a separate module handling all the terminal output, in order to better separate things. At the moment Dub uses the log module from Vibe.d (as I wrote before, which it seems they've just copy-pasted in dub/internal/vibecompat/core/log.d) to print stuff to the terminal, I think replacing it with a module that handles colors and "tags" like in the screenshots I attacched would be the best option. But in order to do something like this cleanly we should first define well how to structure the output and thus the module handling it. Also documenting stuff a bit. Then we'll need to replace all the calls to logInfo, logDiagnostic, logError, etc... in the entire codebase :P A quicker option is to just leave the log calls there, add the escape sequences in order to color the wanted words (as it's currently done in the proof-of-concept) and then fix the Vibe.d log module to handle colors on Windows with the same "workaround" that colorize-d uses. That's a faster way indeed, but a bit dirty IMHO. Also, to handle different color schemes (for MacOS/white background terminals) it may become a mess. So, having a module which handles all the terminal output seems the best option to me, if we want to do stuff cleanly. The main problem is to define its requirements, how it should interface with the rest of the code, how to structure it, etc... then writing the code is the simplest part, as always. I can handle this by myself, but if anyone wants to help that would be really appreciated, especially on planning how to structure the changes. Maybe we can discuss about the implementation on IRC or some other platform?
Jun 08 2018
prev sibling next sibling parent reply Jacob Carlborg <doob me.com> writes:
On 2018-06-08 15:38, Steven Schveighoffer wrote:

 Looks excellent! Two thumbs up from me. Is it cross-platform?
 
 Note on some platforms (ahem, Macos) the background is white, so this 
 should be correctly colored for that possibility.
On macOS everyone should use iTerm :), which has a dark background by default. -- /Jacob Carlborg
Jun 08 2018
next sibling parent Steven Schveighoffer <schveiguy yahoo.com> writes:
On 6/8/18 4:35 PM, Jacob Carlborg wrote:
 On 2018-06-08 15:38, Steven Schveighoffer wrote:
 
 Looks excellent! Two thumbs up from me. Is it cross-platform?

 Note on some platforms (ahem, Macos) the background is white, so this 
 should be correctly colored for that possibility.
On macOS everyone should use iTerm :), which has a dark background by default.
I gotta say, I'm quite satisfied with the default console. I guess I'm not everyone :P -Steve
Jun 08 2018
prev sibling parent Guillaume Piolat <first.last gmail.com> writes:
On Friday, 8 June 2018 at 20:35:50 UTC, Jacob Carlborg wrote:
 On 2018-06-08 15:38, Steven Schveighoffer wrote:

 Looks excellent! Two thumbs up from me. Is it cross-platform?
 
 Note on some platforms (ahem, Macos) the background is white, 
 so this should be correctly colored for that possibility.
On macOS everyone should use iTerm :), which has a dark background by default.
+1
Jun 19 2018
prev sibling parent reply Jacob Carlborg <doob me.com> writes:
On 2018-06-08 15:38, Steven Schveighoffer wrote:

 Note on some platforms (ahem, Macos) the background is white, so this 
 should be correctly colored for that possibility.
I would assume that the theme of the terminal is setup so that all colors (except white and black) work together with the background color of the theme. -- /Jacob Carlborg
Jun 08 2018
parent reply gdelazzari <giacky98.mail gmail.com> writes:
On Friday, 8 June 2018 at 20:41:41 UTC, Jacob Carlborg wrote:
 On 2018-06-08 15:38, Steven Schveighoffer wrote:

 Note on some platforms (ahem, Macos) the background is white, 
 so this should be correctly colored for that possibility.
I would assume that the theme of the terminal is setup so that all colors (except white and black) work together with the background color of the theme.
Actually, I was thinking about that too. In fact, what if a user is using a "classic" dark-background theme on macOS's terminal? Or another terminal which by default uses a dark background, like the one mentioned above? He would get all the colors and the text contrast messed up if I put a different color scheme for macOS only. The only valid option would be to check the background color of the terminal, but I don't think that's possible at all in a standardized way, unless someone can prove me wrong. That would be cool.
Jun 08 2018
next sibling parent gdelazzari <giacky98.mail gmail.com> writes:
For anyone interested, I'm implementing everything in this branch 
of my fork.

https://github.com/gdelazzari/dub/tree/colored-output

Seems like I managed to cleanly substitute the logging module 
with a brand new one :P actually it's most copy-paste with 
changes to allow colored output, but yeah. I also "imported" 
d-colorize (by copying all its file in Dub's codebase - to avoid 
having a Dub package dependency on Dub itself).

This is the relevant part (the new logging module) if anyone 
wants to contribute with feedback or enhancements, since is some 
of the first D code I'm writing. It's still missing the no-TTY 
detection, but it's coming.

https://github.com/gdelazzari/dub/blob/colored-output/source/dub/logging.d
Jun 08 2018
prev sibling parent reply Jacob Carlborg <doob me.com> writes:
On 2018-06-09 00:45, gdelazzari wrote:

 Actually, I was thinking about that too. In fact, what if a user is
 using a "classic" dark-background theme on macOS's terminal? Or another
 terminal which by default uses a dark background, like the one mentioned
 above? He would get all the colors and the text contrast messed up if I
 put a different color scheme for macOS only. The only valid option would
 be to check the background color of the terminal, but I don't think
 that's possible at all in a standardized way, unless someone can prove
 me wrong. That would be cool.
As I mentioned, I think the only way to do this is to avoid using white and black colors and assume all other colors (at least the standard ones) work with the selected theme. For regular text, reset to the default foreground color instead of explicitly using black or white. -- /Jacob Carlborg
Jun 19 2018
parent Anton Fediushin <fediushin.anton yandex.com> writes:
On Tuesday, 19 June 2018 at 19:22:09 UTC, Jacob Carlborg wrote:
 On 2018-06-09 00:45, gdelazzari wrote:

 Actually, I was thinking about that too. In fact, what if a 
 user is
 using a "classic" dark-background theme on macOS's terminal? 
 Or another
 terminal which by default uses a dark background, like the one 
 mentioned
 above? He would get all the colors and the text contrast 
 messed up if I
 put a different color scheme for macOS only. The only valid 
 option would
 be to check the background color of the terminal, but I don't 
 think
 that's possible at all in a standardized way, unless someone 
 can prove
 me wrong. That would be cool.
As I mentioned, I think the only way to do this is to avoid using white and black colors and assume all other colors (at least the standard ones) work with the selected theme. For regular text, reset to the default foreground color instead of explicitly using black or white.
Not just black and white but also some shades of grey. Recently I fixed the same problem in the vibe.d's logger (https://github.com/vibe-d/vibe-core/pull/82) and all I can say is that the only way to deal with colours is to test it on both black and white background.
Jun 19 2018
prev sibling next sibling parent Atila Neves <atila.neves gmail.com> writes:
On Friday, 8 June 2018 at 13:35:36 UTC, gdelazzari wrote:
 Hello everyone, I'm a new user of the language (I've been 
 playing around with it for some months) and I'm really liking 
 it.

 [...]
I like it! Atila
Jun 08 2018
prev sibling next sibling parent Bastiaan Veelo <Bastiaan Veelo.net> writes:
On Friday, 8 June 2018 at 13:35:36 UTC, gdelazzari wrote:
 Hello everyone, I'm a new user of the language (I've been 
 playing around with it for some months) and I'm really liking 
 it.
[...]
 Take a look at these screenshots:

 https://imgur.com/a/3RJyd6m
Nice!!
Jun 08 2018
prev sibling next sibling parent Johannes Loher <johannes.loher fg4f.de> writes:
On Friday, 8 June 2018 at 13:35:36 UTC, gdelazzari wrote:
 Hello everyone, I'm a new user of the language (I've been 
 playing around with it for some months) and I'm really liking 
 it.

 [...]
I really like this very much! I think this a great improvement for dub and I believe ist is very Important for is to get out tooling to be on par with the tools from orher languages. I would love to help you on this. Is there anything in particular you need help with?
Jun 08 2018
prev sibling next sibling parent reply Basile B. <b2.temp gmx.com> writes:
On Friday, 8 June 2018 at 13:35:36 UTC, gdelazzari wrote:
 Hello everyone, I'm a new user of the language (I've been 
 playing around with it for some months) and I'm really liking 
 it.

 [...]

 I started this thread to have a discussion on this before 
 submitting any pull request (which, in the case of this change 
 being apprecciated, I'll happily take the time to make).

 Thanks to anyone in advance,
 Giacomo
While this look okay please in the initial PR don't forget to add code to deactivate colors when DUB will be piped.
Jun 08 2018
parent reply gdelazzari <giacky98.mail gmail.com> writes:
On Friday, 8 June 2018 at 16:11:27 UTC, Basile B. wrote:
 While this look okay please in the initial PR don't forget to 
 add code to deactivate colors when DUB will be piped.
Sure, I won't forget about that. On Linux (and I guess also MacOS) it should be enough to check if stdout is a tty (isatty() from C std) or not, while I don't know how that could be done on Windows. Do you have any idea?
Jun 08 2018
parent reply Basile B. <b2.temp gmx.com> writes:
On Friday, 8 June 2018 at 16:20:12 UTC, gdelazzari wrote:
 On Friday, 8 June 2018 at 16:11:27 UTC, Basile B. wrote:
 While this look okay please in the initial PR don't forget to 
 add code to deactivate colors when DUB will be piped.
Sure, I won't forget about that. On Linux (and I guess also MacOS) it should be enough to check if stdout is a tty (isatty() from C std) or not, while I don't know how that could be done on Windows. Do you have any idea?
Certainly. look at how DMD does that ;)
Jun 08 2018
parent gdelazzari <giacky98.mail gmail.com> writes:
On Friday, 8 June 2018 at 16:33:29 UTC, Basile B. wrote:
 On Friday, 8 June 2018 at 16:20:12 UTC, gdelazzari wrote:
 On Friday, 8 June 2018 at 16:11:27 UTC, Basile B. wrote:
 While this look okay please in the initial PR don't forget to 
 add code to deactivate colors when DUB will be piped.
Sure, I won't forget about that. On Linux (and I guess also MacOS) it should be enough to check if stdout is a tty (isatty() from C std) or not, while I don't know how that could be done on Windows. Do you have any idea?
Certainly. look at how DMD does that ;)
I had a look at it, really helpful, thanks!
Jun 08 2018
prev sibling parent reply Nick Sabalausky <a a.a> writes:
On Friday, 8 June 2018 at 13:35:36 UTC, gdelazzari wrote:
 Take a look at these screenshots:

 https://imgur.com/a/3RJyd6m
Very nice! But related to your motivation for this, I do really wish dub had far less output by default. For example, I don't need to be told over and over that each of my dependencies are up-to-date (or which version was chosen. I can always look at dub.selections.json or use --verbose if I need to check that.) And I dont need to be reminded about --force every time my build succeeds. Or for that matter, be told whether or not the compiler gave an error. If there are errors I can already see that they're there. Etc.
Jun 08 2018
parent gdelazzari <giacky98.mail gmail.com> writes:
On Saturday, 9 June 2018 at 01:17:26 UTC, Nick Sabalausky wrote:
 On Friday, 8 June 2018 at 13:35:36 UTC, gdelazzari wrote:
 Take a look at these screenshots:

 https://imgur.com/a/3RJyd6m
Very nice! But related to your motivation for this, I do really wish dub had far less output by default. For example, I don't need to be told over and over that each of my dependencies are up-to-date (or which version was chosen. I can always look at dub.selections.json or use --verbose if I need to check that.) And I dont need to be reminded about --force every time my build succeeds. Or for that matter, be told whether or not the compiler gave an error. If there are errors I can already see that they're there. Etc.
While I mostly agree with you, I have to note that the reminder that you can use --force to force rebuilding everything only pops up when you "dub build" a package you just built, i.e. if you make some changes to the code and build, it won't show up in the output, as you can see in the first screenshot at my link. I think this is fine and makes sense, since a first time user may try to run "dub build" to rebuild the project without obtaining that effect, and that message will be useful to understand how to actually force rebuilding it if he/she really wants to. I agree that the "Up-to-date" that pops up for every dependency is too verbose, and can definitely be removed. Also the "Failed, dub exited with code X" doesn't make a lot of sense as you said. Also it's not consistent since, if we keep that message, then why not also printing something like "Success, dub exited with code 0, build completed" at the end of a successful build? So yeah, agreed, it should go and doesn't carry more information that what you already know. If anyone has other suggestions regarding Dub's output I'll be happy to take them, since I'm working on that anyway.
Jun 09 2018