digitalmars.dip.ideas - lazy import
- xlogor (35/35) Oct 03 # DIP Idea: Lazy Imports
- Paul Backus (3/8) Oct 03 How will the compiler know whether a symbol from the module is
- xlogor (5/14) Oct 04 Perhaps this could be limited to renamed/static imports?
- Paul Backus (4/19) Oct 04 I suppose that could work.
- monkyyy (4/40) Oct 03 All templates functions already are lazy, the std is slow because
- xlogor (3/57) Oct 04 D compilers lack incremental compilation; lazy evaluation should
- monkyyy (34/44) Oct 04 Memorization only provides speed ups when it effects the whole
- Richard (Rikki) Andrew Cattermole (47/47) Oct 04 A proposal has been made for Python to add lazy imports.
- Quirin Schroll (16/31) Oct 09 **TL;DR:** You need static or selective imports. For “normal”
I’d like to propose a DIP for adding lazy imports to D. D already has the `lazy` keyword for function parameters, which works wonders: - Arguments are only evaluated when (and if) actually used. - This avoids unnecessary work, prevents allocations, and gives a nice middle ground between eager and manual `delegate` passing. It’s a proven feature that shows how well deferred evaluation fits into the language. Currently, though, `import` in D is eager: all imported modules are parsed and semantically analyzed during compilation, regardless of whether their symbols are actually referenced. This can cause unnecessary compile-time costs, especially in large projects with deep dependency graphs. A **lazy import** would extend the same philosophy of deferred evaluation to the module system: the compiler would defer full analysis of a module until it is actually needed (when a symbol from it is referenced). This reduces compile times and makes dependency management more efficient. Proposed Syntax ```d lazy import std.big; ``` This would mean: The compiler does not fully load/parse `std.big` at import time. If no symbols from `std.big` are used, the module never impacts compilation. If symbols are referenced, the compiler processes the module as usual. Alternatively, the same effect could be achieved with an attribute-style approach: ```d lazy import std.big; ```
Oct 03
On Friday, 3 October 2025 at 19:14:00 UTC, xlogor wrote:A **lazy import** would extend the same philosophy of deferred evaluation to the module system: the compiler would defer full analysis of a module until it is actually needed (when a symbol from it is referenced). This reduces compile times and makes dependency management more efficient.How will the compiler know whether a symbol from the module is referenced without parsing and semantically analyzing the module?
Oct 03
On Friday, 3 October 2025 at 19:30:29 UTC, Paul Backus wrote:On Friday, 3 October 2025 at 19:14:00 UTC, xlogor wrote:Perhaps this could be limited to renamed/static imports? ```D lazy import big = std.big; ```A **lazy import** would extend the same philosophy of deferred evaluation to the module system: the compiler would defer full analysis of a module until it is actually needed (when a symbol from it is referenced). This reduces compile times and makes dependency management more efficient.How will the compiler know whether a symbol from the module is referenced without parsing and semantically analyzing the module?
Oct 04
Paul Backus <snarwin gmail.com> On Saturday, 4 October 2025 at 08:54:16 UTC, xlogor wrote:On Friday, 3 October 2025 at 19:30:29 UTC, Paul Backus wrote:I suppose that could work. We could also consider treating renamed and static imports this way by default, without requiring the `lazy` keyword.On Friday, 3 October 2025 at 19:14:00 UTC, xlogor wrote:Perhaps this could be limited to renamed/static imports? ```D lazy import big = std.big; ```A **lazy import** would extend the same philosophy of deferred evaluation to the module system: the compiler would defer full analysis of a module until it is actually needed (when a symbol from it is referenced). This reduces compile times and makes dependency management more efficient.How will the compiler know whether a symbol from the module is referenced without parsing and semantically analyzing the module?
Oct 04
On Friday, 3 October 2025 at 19:14:00 UTC, xlogor wrote:I’d like to propose a DIP for adding lazy imports to D. D already has the `lazy` keyword for function parameters, which works wonders: - Arguments are only evaluated when (and if) actually used. - This avoids unnecessary work, prevents allocations, and gives a nice middle ground between eager and manual `delegate` passing. It’s a proven feature that shows how well deferred evaluation fits into the language. Currently, though, `import` in D is eager: all imported modules are parsed and semantically analyzed during compilation, regardless of whether their symbols are actually referenced. This can cause unnecessary compile-time costs, especially in large projects with deep dependency graphs. A **lazy import** would extend the same philosophy of deferred evaluation to the module system: the compiler would defer full analysis of a module until it is actually needed (when a symbol from it is referenced). This reduces compile times and makes dependency management more efficient. Proposed Syntax ```d lazy import std.big; ``` This would mean: The compiler does not fully load/parse `std.big` at import time. If no symbols from `std.big` are used, the module never impacts compilation. If symbols are referenced, the compiler processes the module as usual. Alternatively, the same effect could be achieved with an attribute-style approach: ```d lazy import std.big; ```All templates functions already are lazy, the std is slow because they get initialized; its a lack of initialization discipline combined with a worse of both worlds policys on templates.
Oct 03
On Friday, 3 October 2025 at 20:18:45 UTC, monkyyy wrote:On Friday, 3 October 2025 at 19:14:00 UTC, xlogor wrote:D compilers lack incremental compilation; lazy evaluation should be explored further.I’d like to propose a DIP for adding lazy imports to D. D already has the `lazy` keyword for function parameters, which works wonders: - Arguments are only evaluated when (and if) actually used. - This avoids unnecessary work, prevents al__cpLocations, and gives a nice middle ground between eager and manual `delegate` passing. It’s a proven feature that shows how well deferred evaluation fits into the language. Currently, though, `import` in D is eager: all imported modules are parsed and semantically analyzed during compilation, regardless of whether their symbols are actually referenced. This can cause unnecessary compile-time costs, especially in large projects with deep dependency graphs. A **lazy import** would extend the same philosophy of deferred evaluation to the module system: the compiler would defer full analysis of a module until it is actually needed (when a symbol from it is referenced). This reduces compile times and makes dependency management more efficient. Proposed Syntax ```d lazy import std.big; ``` This would mean: The compiler does not fully load/parse `std.big` at import time. If no symbols from `std.big` are used, the module never impacts compilation. If symbols are referenced, the compiler processes the module as usual. Alternatively, the same effect could be achieved with an attribute-style approach: ```d lazy import std.big; ```All templates functions already are lazy, the std is slow because they get initialized; its a lack of initialization discipline combined with a worse of both worlds policys on templates.
Oct 04
On Saturday, 4 October 2025 at 08:56:47 UTC, xlogor wrote:On Friday, 3 October 2025 at 20:18:45 UTC, monkyyy wrote:Memorization only provides speed ups when it effects the whole pipeline. "Incremental compilation" which already exists and means a terrible idea where you build libs separately. Push vs pull logistics clash and can easily produce worse of both world results. The c abi is push logistics, theres a push of files you given the compiler with headers and all golbal scope names these get pushed into a pipeline and a executable come out the other end. Templates, out of order evulation and in a sense modules, are pull logistics; modules need to be parsed but a (pure) template is not initialized as a named symbol until used, lerp!Rect maybe be valid but it doesn't exist until used, pulled. --- The std and user base have a mix of takes on either embracing c norms or going with templates, you cant fix this with features. The std needs to have a compilation life time theory(we will see if the current plan improve things, Im mildly hopeful that the core vs upper end split will work by accident, but the core team are anti template) The user base needs education or do find out what the hell these 90 minute compiles from a small minority come from. --- We should go full "pull"; and incremental compilation is a "push" (anti-)feature. All imports are already lazy as they can be.All templates functions already are lazy, the std is slow because they get initialized; its a lack of initialization discipline combined with a worse of both worlds policys on templates.D compilers lack incremental compilation; lazy evaluation should be explored further.lazy import big = std.big;If you want a lazy namespace so a import doesnt exist until called you could try ```d template big(){ import big; } ``` Then you call `big!().writeln;` but this breaks the usual overload rules and ufcs so it cant be made a default behavior.
Oct 04
A proposal has been made for Python to add lazy imports. https://pep-previews--4622.org.readthedocs.build/pep-0810/ These appear to be equivalent to our static imports + only evaluate that module on use. ----- If a module is passed on the command line (and is a root), it cannot be lazy. It will be semantically analyzed in order. This could only apply to import paths. Furthermore the way the compiler works is in stages, a function body will be analyzed either when its required or after everything else in the module has been. For imports this extra step for function bodies is meant to be elided. For this reason I see minimal benefit as it currently stands, given the above definitions. Proof: ```d module def; void func() { sf; } ``` ```d module app; import def; void main() { func; } ``` ``` $ dmd main.d main.obj : error LNK2019: unresolved external symbol _D3def4funcFZv referenced in function _Dmain main.exe : fatal error LNK1120: 1 unresolved externals Error: linker exited with status 1120 C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.41.34120\bin\HostX64\x64\link.exe /NOLOGO "main.obj" /DEFAULTLIB:phobos64 /LIBPATH:"C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.41.34120\lib\x64" legacy_stdio_definitions.lib /LIBPATH:"C:\Program Files (x86)\Windows Kits\10\Lib\10.0.20348.0\ucrt\x64" /LIBPATH:"C:\Program Files (x86)\Windows Kits\10\lib\10.0.20348.0\um\x64" Error: undefined reference to `void def.func()` referenced from `_Dmain` perhaps `.d` files need to be added on the command line, or use `-i` to compile imports ```
Oct 04
On Friday, 3 October 2025 at 19:14:00 UTC, xlogor wrote:A **lazy import** would extend the same philosophy of deferred evaluation to the module system: the compiler would defer full analysis of a module until it is actually needed (when a symbol from it is referenced). This reduces compile times and makes dependency management more efficient. Proposed Syntax ```d lazy import std.big; ``` This would mean: The compiler does not fully load/parse `std.big` at import time. If no symbols from `std.big` are used, the module never impacts compilation. If symbols are referenced, the compiler processes the module as usual.**TL;DR:** You need static or selective imports. For “normal” imports, it can’t be done. Any symbol that’s encountered could be from the module. Without parsing that, it cannot be ruled out. Even if it is found in another module, `std.big` could contain the viable overloads (ideally the only ones, then, to avoid an error). It can only work if that makes `std.big` implicitly a static import. I’d suggest `lazy` requires `static` in that case, so it’s clearer. Such a restriction could be lifted later. Another option is to require selective imports. Contrary to `import std.big` (be it lazy or not), in `import std.big : stick`, it can be determined if a symbol referenced might be imported from `std.big` since you stated in the importing module what you import from `std.big`. Only if you end up “using” one of the symbols, the module has to be parsed.
Oct 09