digitalmars.D.announce - Redub v1.17.1: Walking on its own direction
- Hipreme (230/233) Nov 19 After redub being in the shadow of dub on its older versions,
- Anton Pastukhov (4/7) Nov 20 Unfortunately all links are broken. And this is the actual redub
- Hipreme (15/22) Nov 20 Shoot, sorry, I'll post all the link again.
- jmh530 (2/3) Nov 20 I haven't tried this yet, but thanks for all the work on this.
- singingbush (5/9) Nov 25 Nice. In terms of other plugins, I'd be interested in having code
- Hipreme (21/30) Nov 27 That is quite simple with the new plugin situation on redub. The
After redub being in the shadow of dub on its older versions,
redub has now took a step forward, going in a new direction
bringing to the table new features that were not inside dub.
For that to happen, redub has now severed its ties with dub by
removing it completely from its code. For accomplishing that,
redub did:
- Implemented dub registry package fetching code and simulated
dub behavior
- Implemented SDL -> JSON parser
- Implemented --single file parser
Redub is now able to automatically update itself, build itself
and replace itself by calling `redub update`. If you use it from
a redub repo, it will git pull, if you do it from a .exe file, it
will download the latest release tag and build it. Your current
redub.exe will be replaced at the end, guaranteeing instantaneous
updates.
You won't need to use `rdmd` anymore as redub seeks for more
performance and flexibility. With redub plugins, you won't be
able to keep writing platform specific code as you can just use
the old D code you have and directly interface with redub
painlessly.
Redub will also fetch your dependencies in parallel making it
even faster
I was able to reduce from 180ms to 30ms one of the projects here
in dub, so, it is really way better
-- Also, I'm testing right now my new tool for automatically
generating changelog, be sure to check its capabilities right in
the footer of this post
- `redub update` command - Automatically updates redub to the
latest version on your system
- Fetch: Redub is now able to fetch without dub.
- Fetch: Redub now fetches in parallel for newer packages,
bringing a speed gain up to 80% compared to the single thread one.
- Fully added --single support for redub, with advanced caching
support
- SDL->JSON parser to Redub, removing one more dependency from
dub
- Support for wasm/webassembly OS and now the json cache is
cleared after parsing a project, hide cache write log if it is
too small
- 'darwin' platform filter, fixed run execution by escaping its
command
- Filter out hidden files from adv cache and fixed cache
invalidation for linking step
- Caching solution with print-ups with copy enough and
up-to-date builds
- Use hardlink instead of copy making it much faster, also added
cache for linker
- Initial binary caching
- pkg-config on posix
- Pkg-config execution is fully parallelized, added more timings
on vverbose mode
- defaultCompiler, compilers cache for faster inference, and
Redub has now new additions, starting from v1.14.0. Those are
called **plugins**.
For using it, on the JSON, you **must** specify first the plugins
that are usable. For doing that, you need to add:
```json
"plugins": {
"getmodules": "C:\\Users\\Hipreme\\redub\\plugins\\getmodules"
}
```
That line will both build that project and load it inside the
registered plugins (That means the same name can't be specified
twice)
The path may be either a .d module or a dub project
WARNING: This may be a subject of change and may also only
support redub projects in the future, since that may only
complicate code with a really low gain
Redub will start distributing some build plugins in the future.
Currently, getmodules plugins is inside this repo as an example
only but may be better used.
Only preBuild is currently supported since I haven't found
situations yet to other cases.
For it to be considered a redub plugin to be built, that is the
minimal code:
```d
module getmodules;
import redub.plugin.api;
class GetModulePlugin : RedubPlugin {}
mixin PluginEntrypoint!(GetModulePlugin);
```
For using it on prebuild, you simply specify the module and its
arguments:
```json
"preBuildPlugins": {
"getmodules": ["source/imports.txt"]
}
```
- [**GetModule plugin**](./plugins/getmodules/source/getmodules.d)
- [**Example Usage**](./tests/plugin_test/dub.json)
- Redub now includes version of the compiler used to be built,
throws an exception if an inexistent configuration is sent,
defaults plugin compiler to be the same as the used one to build
compiler with arch not triggering LDC anymore. Fixed issue where
specifying global compiler would fai
- Plugins won't include environment variables in the process of
building itself
- Do not output deps by default, this is causing compile time
slowdown rather than good
- Less memory allocation inside adv_diff and inference for
simplified hashing when file bigger than 2MB
- Added simplified hashing for output files bringing a much
afster cache writing
- Faster copy cache formula using a simplified process of
checking
- Fixed dynamic lib name for linux and osx
- Fix linux build
- Adv diff will now use content hash for cache formulas instead
of modification time, this will fix a problem where it wasn't
able to get name changes on file
- Remove start group from macOS
- Now redub will always specify to handle circular linker
dependencies
- Fixed some cases where a dependency could be double registered
when using subpackages
- Fixed postBuildCommands to be used after .exe gen
- Fixed on can't build for windows when having different drive
disks
- Improved logging to print less things, also fixed an issue
where sometimes a sub package would get a wrong name
- Fixed problem getting wrong cache when cross compiling, fixed
the problem where the run arguments were not being sent correctly
- Added getOutputName for redub api, added verbose build type
- Filter -preview= from the dflags
- Added time-trace and mixin-check to the build
- Improvements on redub library api
- Improved build to save cache even when it fails
- Improved source imports
- API should now get ISA for building
- Minor performance improvement
- Improved a little more the compiler finding so it doesn't try
to search always on global path
- Reduced memory allocations inside adv_diff and improved
hexstring parsing algorithm
- Improved compilation API so now it doesn't require a full
cache status
- Improve performance of diffstatus
- Improvement for showing which file is dirty
- Improve info on which files weren't found on adv_diff and
initial deps parsing
- Added targetPath and targetName from CLI, while keeping the
cache calculation smarter
- Improved redub clean to try cleaning all the files that were
generated in the process
only occurs before building since it gives the opportunity to
change the tree
- Now the cache is being able to be calculated in parallel
- Auto clear compiler cache when changing redub version
- Smarter log in every build type
- Made the caching calculation be done in a separate thread for
static libraries, making the program fully efficient
- Better support on preGenerateCommands + handling --single +
more environment being parsed
- Better error information when incompatible semver is found
- targetName is now being correctly used
- Support for linux
- Remove comments from SDL before converting to JSON, handle
buildTypes correctly from SDL converter, clean test files
- Pending merge requirements weren't being processed with
environment
requirement time, removed it from build requirements, copy
attributes on dir copy, fixed target name
- Dynamic library names
- Crash on empty adv cache formula and redub plugins are now
global
- Correctly infer staticLibrary for non root packages
- A problem where some files would not be copied
- Clean will now also clear the cache folder so it will always
cause a rebuild (solves the trying to run inexistent file
problem), and now also outputs libraries when they are root
- Now the copy cache will also look for object files
- Bug wehre some times the creation of cache files would not
happen and still would cache the file
- Bug on LDC where it would not allow to build if the directory
for object dir didn't exist
- Verbose output for posix
- Regression bug on compiling for posix
- Checking compiler for null return
- satisfies now correctly uses matchAll
- Even more speed on the up to date status
- old build warnings
- Linking on LDC, LDC not polluting anymore user folder,
simplified compilation API, added clean on force builds
- Cache for the new entries
- Removed loggings
- Windows is now using correct d_dependencies
- Not working on windows d_dependencies
- d_dependencies received in some cases more information that
wasn't being handled
- Bug where sometimes it would incorrectly identify as up to date
- subPackages would sometimes not get the correct version
- Now supporting glob matching copy on copyFilse
- index out of range
- Now none projects should not build and only run its commands.
Added invalid targetType and fixed preGenerateCommands when
inside configuration
- Now the link files always are sent to their parents
- Now differing dependency version from different depth level
are updated
- Now correctly adding environment variables per project
- Redub now handles up to date builds much better
- copyDir now make create a directory
- Bug when pkg-config does not exists
- Now sourceLibrary with dependencies can be handled better
- Wrong bug which made not be able to build single projects
- More flexibility when assigning a defaultCompiler
- Faster dependency resolution by logging less information
- Build issue on linux
- Made the output shorter when uusing up to date builds [ignore
time took]
|Changelog Metadata|
|------------------:|
|Generated with
[changelog-d](https://code.dlang.org/packages/changelog_d) |
|Contribute at https://github.com/MrcSnm/changelog-d|
Nov 19
On Wednesday, 20 November 2024 at 02:18:07 UTC, Hipreme wrote:After redub being in the shadow of dub on its older versions, redub has now took a step forward, going in a new direction bringing to the table new features that were not inside dub.Unfortunately all links are broken. And this is the actual redub project link: https://github.com/MrcSnm/redub Never heard of it before, looks interesting
Nov 20
Hipreme <msnmancini hotmail.com> On Wednesday, 20 November 2024 at 08:29:22 UTC, Anton Pastukhov wrote:On Wednesday, 20 November 2024 at 02:18:07 UTC, Hipreme wrote:Shoot, sorry, I'll post all the link again. https://code.dlang.org/packages/redub): You can run with `dub run redub`. Though I would recommend building it and adding to PATH since it will run faster :) https://github.com/MrcSnm/redub): Open issues or pull requests plugin](https://github.com/MrcSnm/redub/blob/main/plugins/getmodules/s urce/getmodules.d): Add to your build process a step of generating a file containing all the modules Usage](https://github.com/MrcSnm/redub/blob/main/tests/plugin_test/dub.json): How to use that as a plugin instead of an external script [changelog-d package](https://code.dlang.org/packages/changelog-d): Package used to generate this changelingAfter redub being in the shadow of dub on its older versions, redub has now took a step forward, going in a new direction bringing to the table new features that were not inside dub.Unfortunately all links are broken. And this is the actual redub project link: https://github.com/MrcSnm/redub Never heard of it before, looks interesting
Nov 20
On Wednesday, 20 November 2024 at 02:18:07 UTC, Hipreme wrote:[snip]I haven't tried this yet, but thanks for all the work on this.
Nov 20
On Wednesday, 20 November 2024 at 02:18:07 UTC, Hipreme wrote:After redub being in the shadow of dub on its older versions, redub has now took a step forward, going in a new direction bringing to the table new features that were not inside dub. ...Nice. In terms of other plugins, I'd be interested in having code signing and convenient packaging so that simply running the build tool can produce an RPM for Fedora. Calling through to external tools is fine.
Nov 25
On Monday, 25 November 2024 at 19:21:36 UTC, singingbush wrote:On Wednesday, 20 November 2024 at 02:18:07 UTC, Hipreme wrote:That is quite simple with the new plugin situation on redub. The good thing is that it is easily extensible for any user case one may have. One of the reasons I've done plugins is that the way to achieve that in dub, is that you would only be able to spawn processes which is slower and platform dependent. While with plugins, you can write D code, modify some of your build recipe inside the plugin AND still get cache working. Continuing the announce: Redub v1.19.0 has also improved how the cache is stored. From my engine, I was able to reduce it from 550KB to 78KB. That difference can make a lot of difference in people using HD instead of SSD and also improve the JSON parsing speed. Beyond that, I've also reduced the syscalls when comparing hashes, thus making it even faster (I got almost a 30% speedup when doing that) Some parts of the code were made nogc which also reduced a bunch of allocations. In the end, both up to date builds AND non up to date builds got way faster on the caching side, so, even slower processors can now handle the work just fine.After redub being in the shadow of dub on its older versions, redub has now took a step forward, going in a new direction bringing to the table new features that were not inside dub. ...Nice. In terms of other plugins, I'd be interested in having code signing and convenient packaging so that simply running the build tool can produce an RPM for Fedora. Calling through to external tools is fine.
Nov 27