www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.ide - Dustmite + Visual D

reply Michelle Long <HappyDance321 gmail.com> writes:
C:\a\b\b\b.dustmite\..\..\..\..\x\y\pegged\dynamic\grammar.d: The 
process cannot access the file because it is being used by 
another process

When I run dustmite on a project.

The x is a junction. Nothing is accessing grammar.d except maybe 
visual studio internally... but even if it were it still should 
be accessible for copying. (after all I can open the file).

Seems visual D craps out because it is a junction?

Could this be remedied?

I use junctions often because it helps with organization(rather 
than duplicating).

It may not be from it being a junction, but if it is not I can't 
imagine why it is having trouble with it(assuming the error 
message is correct).
Nov 04 2018
next sibling parent reply Vladimir Panteleev <thecybershadow.lists gmail.com> writes:
On Sunday, 4 November 2018 at 17:16:55 UTC, Michelle Long wrote:

 C:\a\b\b\b.dustmite\..\..\..\..\x\y\pegged\dynamic\grammar.d: 
 The process cannot access the file because it is being used by 
 another process

 When I run dustmite on a project.


As the error message says, the problem occurs because another 
file is accessing it.

Some common culprits are:

- Anti-virus / security software, such as Windows Defender. 
Adding the directory containing the D code being reduced as an 
exception usually helps.
- File synchronization software, such as OneDrive / SyncThing.
- Editors/IDEs. Make sure the directory being reduced is not part 
of any project/solution open in Visual Studio etc.

I think I made Dustmite retry after a second when it gets that 
error. Is that not the case? Post the full output, if so.
Nov 04 2018
parent reply Michelle Long <HappyDance321 gmail.com> writes:
On Sunday, 4 November 2018 at 17:20:41 UTC, Vladimir Panteleev 
wrote:

 On Sunday, 4 November 2018 at 17:16:55 UTC, Michelle Long wrote:

 C:\a\b\b\b.dustmite\..\..\..\..\x\y\pegged\dynamic\grammar.d: 
 The process cannot access the file because it is being used by 
 another process

 When I run dustmite on a project.


 As the error message says, the problem occurs because another 
 file is accessing it.


No, have you not learned in your life time that just because 
something or someeone says something doesn't make it true?


 Some common culprits are:

 - Anti-virus / security software, such as Windows Defender. 
 Adding the directory containing the D code being reduced as an 
 exception usually helps.
 - File synchronization software, such as OneDrive / SyncThing.
 - Editors/IDEs. Make sure the directory being reduced is not 
 part of any project/solution open in Visual Studio etc.


Nope, none of this because I run none. Please trust people a bit 
more when they say what they say. Not everyone is an imbecile.


 I think I made Dustmite retry after a second when it gets that 
 error. Is that not the case? Post the full output, if so.



This has nothing to do with dustmite as far as I know. That is 
the full output.

Visual D creates a temp project and when it tries to copy the 
files this error occurs. It is almost surely due to a junction 
issue as I mentioned. (believe me when I tell you I'm not a moron 
so when I say something it generally has some truth in it)

If this is dustmite and not visual D then it most likely is due 
to the junction.


I have used dustmite before and it has worked in visual D, but 
either a regression occurred or it is due to using the 
junction(which is the first project I have created that uses a 
junction and dustmited).


Let me enlighten you a little: The reason why this is most likely 
a junction is:

1. There is little else it could be except visual studio itself 
locking the file, which it doesn't seem to be and has worked 
before.

2. Junctions are known to cause problems with some copy routines 
because one can't simply copy a junction as normal. The reason 
being is the junction suppose to be duplicated or all the data?



e.g., this happens to others:

https://answers.microsoft.com/en-us/windows/forum/windows8_1-files/bug-file-copies-freeze-when-symbolic-links-or/10e0c4f3-ca9e-403e-ba53-58be6bddce36?auth=1
Nov 04 2018
parent reply Vladimir Panteleev <thecybershadow.lists gmail.com> writes:
On Sunday, 4 November 2018 at 21:11:27 UTC, Michelle Long wrote:

 No, have you not learned in your life time that just because 
 something or someeone says something doesn't make it true?


In all my years of using Windows, I have never seen or heard for 
this error message to lie (i.e. occur due to a reason different 
than what it states). I've also used junctions and symbolic links 
a lot.

It's possible the junction is confusing some software into not 
closing a file, though.


 Nope, none of this because I run none. Please trust people a 
 bit more when they say what they say. Not everyone is an 
 imbecile.


Sometimes, the reason can be really unobvious. We've ran into a 
similar problem a while ago on CI machines, which turned out to 
be the security software included with the then-new version of 
Windows. It doesn't hurt to check.


 This has nothing to do with dustmite as far as I know. That is 
 the full output.


Probably trying to invoke Dustmite manually would be progress 
then, with the Visual D bug report aside.


 If this is dustmite and not visual D then it most likely is due 
 to the junction.


1. DustMite will not touch anything outside the parent directory 
of the path you give it (though the test command might).
2. DustMite doesn't care about junctions and treats them as 
directories. When creating test copies of the source, it will 
recreate the directory structure with junctions replaced with 
directories.


 1. There is little else it could be except visual studio itself 
 locking the file, which it doesn't seem to be and has worked 
 before.


FWIW, you can use Process Explorer (by Microsoft) or Process 
Hacker (FOSS) to find which processes hold handles to a file. 
That might provide a clue.


 2. Junctions are known to cause problems with some copy 
 routines because one can't simply copy a junction as normal. 
 The reason being is the junction suppose to be duplicated or 
 all the data?


Yes, well known gotcha. std.file.dirEntries has a followSymlink 
parameter. DustMite uses the default (true).
Nov 04 2018
parent reply Michelle Long <HappyDance321 gmail.com> writes:
On Sunday, 4 November 2018 at 22:12:16 UTC, Vladimir Panteleev 
wrote:

 On Sunday, 4 November 2018 at 21:11:27 UTC, Michelle Long wrote:

 No, have you not learned in your life time that just because 
 something or someeone says something doesn't make it true?


 In all my years of using Windows, I have never seen or heard 
 for this error message to lie (i.e. occur due to a reason 
 different than what it states). I've also used junctions and 
 symbolic links a lot.

 It's possible the junction is confusing some software into not 
 closing a file, though.


 Nope, none of this because I run none. Please trust people a 
 bit more when they say what they say. Not everyone is an 
 imbecile.


 Sometimes, the reason can be really unobvious. We've ran into a 
 similar problem a while ago on CI machines, which turned out to 
 be the security software included with the then-new version of 
 Windows. It doesn't hurt to check.


 This has nothing to do with dustmite as far as I know. That is 
 the full output.


 Probably trying to invoke Dustmite manually would be progress 
 then, with the Visual D bug report aside.


 If this is dustmite and not visual D then it most likely is 
 due to the junction.


 1. DustMite will not touch anything outside the parent 
 directory of the path you give it (though the test command 
 might).
 2. DustMite doesn't care about junctions and treats them as 
 directories. When creating test copies of the source, it will 
 recreate the directory structure with junctions replaced with 
 directories.


 1. There is little else it could be except visual studio 
 itself locking the file, which it doesn't seem to be and has 
 worked before.


 FWIW, you can use Process Explorer (by Microsoft) or Process 
 Hacker (FOSS) to find which processes hold handles to a file. 
 That might provide a clue.


 2. Junctions are known to cause problems with some copy 
 routines because one can't simply copy a junction as normal. 
 The reason being is the junction suppose to be duplicated or 
 all the data?


 Yes, well known gotcha. std.file.dirEntries has a followSymlink 
 parameter. DustMite uses the default (true).



I am pretty sure it is a bug in Visual D not handling the 
junction right. It was a fresh boot with no other programs ran 
before and given other experiences it seems almost surely(95%+) 
that it is a junction problem with visual D. Since it is such a 
high liklihood it should be the first thing tried.

While I could run dustmite from the command line, the point of 
using visual D is because it handles it all automatically, which 
is nice, when it works.

Now, I can't be sure if it's Visual D or Dustmite because that is 
the only error I get, but visual D says it needs to copy the 
project to a temp folder and then the error appears immediately. 
I bet if I replaced the junction with a normal directory it would 
work but then that defeats the purpose of using a junction.

It would take Rainer just a few minutes max to determine this by 
simply created a project or opening a pre-existing one and adding 
a junction the dragging it to the project and adding a d file in 
it then trying the dustmite command.

Again, I do not think this is a dustmite problem, it occurs when 
trying to run dustmite using the Visual D dustmite command(which, 
again, has worked in the past without issue and all it does is 
copy the project then run dustmite... and the error is about 
accessing a file).

Because I'm sure this is related to Visual D and 
junctions(whatever copying method it uses) I'm willing not to try 
random stuff to try and fix it... expecting that Rainer can 
reproduce this problem fairly quickly when he gets time. If he 
adds a junction and it doesn't have an error then I will look in 
to it more on my side. No reason for me to spend hours trying to 
figure out possible edge cases until I know it is not the most 
probable cause. It will take him a few minutes to investigate and 
chances are it will be easily solved.

If it turns out something is locking the file, then that is still 
a problem because nothing I have done locks the file it complains 
on(so it's still a bug). I did not ever touch that file in any 
way. I simply loaded the project and ran the command(the file is 
not even loaded in the editor or being accessed by anything 
external(unless windows is doing funky shit, which I doubt since 
I disabled most of all the crap it does)).
Nov 04 2018
parent reply Rainer Schuetze <r.sagitario gmx.de> writes:
On 05/11/2018 00:08, Michelle Long wrote:

 It would take Rainer just a few minutes max to determine this by simply
 created a project or opening a pre-existing one and adding a junction
 the dragging it to the project and adding a d file in it then trying the
 dustmite command.


I just tried that and it worked fine for my tiny example. Copying the
project is just a series of

	std.file.mkdirRecurse(dirName(destname));
	std.file.copy(srcname, destname);

on the files in the project.

Do you get the message in the Visual D output pane or the console window?
Nov 04 2018
parent reply Michelle Long <HappyDance321 gmail.com> writes:
On Monday, 5 November 2018 at 07:24:36 UTC, Rainer Schuetze wrote:

 On 05/11/2018 00:08, Michelle Long wrote:

 It would take Rainer just a few minutes max to determine this 
 by simply created a project or opening a pre-existing one and 
 adding a junction the dragging it to the project and adding a 
 d file in it then trying the dustmite command.


 I just tried that and it worked fine for my tiny example. 
 Copying the project is just a series of

 	std.file.mkdirRecurse(dirName(destname));
 	std.file.copy(srcname, destname);

 on the files in the project.

 Do you get the message in the Visual D output pane or the 
 console window?


It is a visual D output problem.

I tried several things and had strange results.

I created a new project in D and ran dustmite:

created clean copy of the project in 
c:\users\main\documents\visual studio 
2017\Projects\WindowsApp4\WindowsApp4.dustmite
Failed to make stdin stream inheritable by child process (Access 
is denied.)


I tried removing the junction and it still seems to fail on 
grammar.d.

I ran a chkdsk and no errors.


I used lockhunter and no locks on the file.

I tried copying the project over manually and no files where 
copied from the junction.

I tried copying the junction directory and everything was copied 
but pegged.

the dustmite directory shows that it is copying(some files)

So this might not be a Visual D issue but something in the OS 
itself.


The error I get is

D:\.......\Libraries\pegged\dynamic\grammar.d: The process cannot 
access the file because it is being used by another process.

In visual D's error pane.

Actually, the error is:

D:\Projects\Game\Game\Game.dustmite\..\..\..\..\D\Libraries\pegge
\dynamic\grammar.d: The process cannot access the file because it is being used
by another process.

When I click on the error it opened the file in the editor and it 
was correct.


I really have no idea what is going on.. maybe the junction is 
"bad"(it works in xyplorer but maybe something got corrupted?).

The fact that copying the project dir manually using xyplorer or 
windows file explorer both fail to copy the junction suggest it 
is not a Visual D problem.

When I copy the files over from the library directly(bypassing 
the junction issue) the same problem occurs.

So, it would suggest a problem with the grammar.d file, yet I can 
read it just fine.

Maybe it is just a problem with that file in relation to Visual D?

Maybe if you could allow the process to continue if we ignore 
"Remove dustmite dir" then I could just copy the files 
manually(which works).

It probably won't fix the stdin issue but it would get around the 
copying issue.

Other than that, I'm at a loss.

I moved some stuff around trying to see if it was anything with 
the project and all it it was change the copy error:

D:\Projects\Game\Game\Game.dustmite\..\..\..\Libraries\arsd\png.d: The process
cannot access the file because it is being used by another process.

It did change the path but probably because I pointed the folder 
in visual studio to the junction(I think I was actually using the 
library path before).

In any case it seems maybe something is causing the file to be 
temporarily locked inside visual study when dustmite command is 
issued.

Could this be with the Visual D copying method?

If you are copying each file using recursion maybe it is 
accessing the file to copy before it is released in "looking" up?

You could try an alternative copy such as fastcopy(which can 
handle junctions well)?

If I click on the file it says can't open(such as png.d) then it 
opens it in the editor, so the path is correct and the file is 
not locked then.

Maybe retry the copy with a sleep a few times if it fails at 
first?
Nov 05 2018
parent reply Rainer Schuetze <r.sagitario gmx.de> writes:
On 06/11/2018 04:09, Michelle Long wrote:

 D:\Projects\Game\Game\Game.dustmite\..\..\..\..\D\Libraries\pegged\dynamic\grammar.d:
 The process cannot access the file because it is being used by another
 process.
 


I can reproduce it now, but it has nothing to do with links/junctions.

The problem is that the shown file is outside of the project folder
hierarchy. Visual D tries to recreate the projects folder structure, so
it copies

D:\Projects\Game\Game\Game\..\..\..\..\D\Libraries\pegged\dynamic\grammar.d

to

D:\Projects\Game\Game\Game.dustmite\..\..\..\..\D\Libraries\pegged\dynamic\grammar.d

which is the same file!

A simple workaround is to use a junction in the project folder to access
pegged.

I guess Visual D should not copy files outside of the project folder,
but they are also not subject to dustmite in that case.

Alternatively, the dustmite directory could be created in a common
parent folder for all files, but that doesn't work for files distributed
over different drives.
Nov 07 2018
parent reply Michelle Long <HappyDance321 gmail.com> writes:
On Thursday, 8 November 2018 at 07:16:19 UTC, Rainer Schuetze 
wrote:

 On 06/11/2018 04:09, Michelle Long wrote:

 D:\Projects\Game\Game\Game.dustmite\..\..\..\..\D\Libraries\pegged\dynamic\grammar.d:
 The process cannot access the file because it is being used by 
 another
 process.
 


 I can reproduce it now, but it has nothing to do with 
 links/junctions.

 The problem is that the shown file is outside of the project 
 folder hierarchy. Visual D tries to recreate the projects 
 folder structure, so it copies

 D:\Projects\Game\Game\Game\..\..\..\..\D\Libraries\pegged\dynamic\grammar.d

 to

 D:\Projects\Game\Game\Game.dustmite\..\..\..\..\D\Libraries\pegged\dynamic\grammar.d

 which is the same file!

 A simple workaround is to use a junction in the project folder 
 to access pegged.

 I guess Visual D should not copy files outside of the project 
 folder, but they are also not subject to dustmite in that case.

 Alternatively, the dustmite directory could be created in a 
 common parent folder for all files, but that doesn't work for 
 files distributed over different drives.


whoo! ;) That dogged a bullet! Glad you were able to figure it 
out.

I am using a junction!

I have my libs in D\Libraries.

In my project I have a junction that points to it.

I dragged that junction to VS to add the files.

So they don't exist at the same place.

Visual D seems to copy some of the files.

Maybe the better solution would just be to delay and repeat.

If somehow it is copying two files simultaneously then the delay 
and retry might a few times will just copy it twice(it would have 
to override it).

Or maybe just check if the file already exists and avoid copying 
it? (although the grammar.d file that it complained about was 
never copied, so this probably won't work)

Maybe you could simply record any failures and try them at the 
end.

Here I'm assuming the locking is temporary and will disappear 
quickly enough given that I can never see the file being locked.
Nov 08 2018
parent reply Rainer Schuetze <r.sagitario gmx.de> writes:
On 08/11/2018 10:43, Michelle Long wrote:

 On Thursday, 8 November 2018 at 07:16:19 UTC, Rainer Schuetze wrote:

 On 06/11/2018 04:09, Michelle Long wrote:

 D:\Projects\Game\Game\Game.dustmite\..\..\..\..\D\Libraries\pegged\dynamic\grammar.d:

 The process cannot access the file because it is being used by another
 process.


 I can reproduce it now, but it has nothing to do with links/junctions.

 The problem is that the shown file is outside of the project folder
 hierarchy. Visual D tries to recreate the projects folder structure,
 so it copies

 D:\Projects\Game\Game\Game\..\..\..\..\D\Libraries\pegged\dynamic\grammar.d


 to

 D:\Projects\Game\Game\Game.dustmite\..\..\..\..\D\Libraries\pegged\dynamic\grammar.d


 which is the same file!

 A simple workaround is to use a junction in the project folder to
 access pegged.

 I guess Visual D should not copy files outside of the project folder,
 but they are also not subject to dustmite in that case.

 Alternatively, the dustmite directory could be created in a common
 parent folder for all files, but that doesn't work for files
 distributed over different drives.


 
 whoo! ;) That dogged a bullet! Glad you were able to figure it out.
 
 I am using a junction!
 
 I have my libs in D\Libraries.
 
 In my project I have a junction that points to it.
 
 I dragged that junction to VS to add the files.
 
 So they don't exist at the same place.
 
 Visual D seems to copy some of the files.
 
 Maybe the better solution would just be to delay and repeat.
 
 If somehow it is copying two files simultaneously then the delay and
 retry might a few times will just copy it twice(it would have to
 override it).
 
 Or maybe just check if the file already exists and avoid copying it?
 (although the grammar.d file that it complained about was never copied,
 so this probably won't work)
 
 Maybe you could simply record any failures and try them at the end.
 
 Here I'm assuming the locking is temporary and will disappear quickly
 enough given that I can never see the file being locked.
 
 


I suspect there is still a misunderstanding: your project very likely
still refers to these files via a relative path, not through the
junction. You can verify this by looking at the list of files at the end
of the .visualdproj file with a text editor.
Nov 08 2018
parent reply Michelle Long <HappyDance321 gmail.com> writes:
 I suspect there is still a misunderstanding: your project very 
 likely still refers to these files via a relative path, not 
 through the junction. You can verify this by looking at the 
 list of files at the end of the .visualdproj file with a text 
 editor.



Ok, you are right. I think i might have dragged some items in 
manually from the file explorer after I removed them thinking I 
didn't need them after I dragged in the junction.

I noticed this:

     <File path="Libraries\arsd\color.d" />
     <File path="Libraries\arsd\gamehelpers.d" />
     <File path="..\..\..\Game\Game\Libraries\arsd\png.d" />
     <File path="Libraries\arsd\simpledisplay.d" />
     <File path="Libraries\arsd\stb_truetype.d" />

(note that they are all the same paths!)

In any case, it should be an easy fix either way?

Maybe the relative paths are causing D's file routines to crap 
out?
Nov 08 2018
parent reply Michelle Long <HappyDance321 gmail.com> writes:
On Thursday, 8 November 2018 at 23:09:53 UTC, Michelle Long wrote:

 I suspect there is still a misunderstanding: your project very 
 likely still refers to these files via a relative path, not 
 through the junction. You can verify this by looking at the 
 list of files at the end of the .visualdproj file with a text 
 editor.



 Ok, you are right. I think i might have dragged some items in 
 manually from the file explorer after I removed them thinking I 
 didn't need them after I dragged in the junction.

 I noticed this:

     <File path="Libraries\arsd\color.d" />
     <File path="Libraries\arsd\gamehelpers.d" />
     <File path="..\..\..\Game\Game\Libraries\arsd\png.d" />
     <File path="Libraries\arsd\simpledisplay.d" />
     <File path="Libraries\arsd\stb_truetype.d" />

 (note that they are all the same paths!)

 In any case, it should be an easy fix either way?

 Maybe the relative paths are causing D's file routines to crap 
 out?


So, this is definitely the reason for the errors. I manually 
removed the ..\'s and was able to get the project to copy.

Note that when the files are separately dragged in they can 
produce different results.

The paths are valid but but it seems like the copy routines do 
not understand relative and mixed paths.

Now I get

created clean copy of the project in 
D:\Projects\Game\Game\Game.dustmite
Failed to make stdin stream inheritable by child process (Access 
is denied.)

so now getting that error about stdin that I get from a fresh 
project... this seems like a bug in Visual D too.
Nov 08 2018
parent Rainer Schuetze <r.sagitario gmx.de> writes:
On 09/11/2018 01:57, Michelle Long wrote:

 On Thursday, 8 November 2018 at 23:09:53 UTC, Michelle Long wrote:

 I suspect there is still a misunderstanding: your project very likely
 still refers to these files via a relative path, not through the
 junction. You can verify this by looking at the list of files at the
 end of the .visualdproj file with a text editor.



 Ok, you are right. I think i might have dragged some items in manually
 from the file explorer after I removed them thinking I didn't need
 them after I dragged in the junction.

 I noticed this:

     <File path="Libraries\arsd\color.d" />
     <File path="Libraries\arsd\gamehelpers.d" />
     <File path="..\..\..\Game\Game\Libraries\arsd\png.d" />
     <File path="Libraries\arsd\simpledisplay.d" />
     <File path="Libraries\arsd\stb_truetype.d" />

 (note that they are all the same paths!)

 In any case, it should be an easy fix either way?

 Maybe the relative paths are causing D's file routines to crap out?


 
 So, this is definitely the reason for the errors. I manually removed the
 ..\'s and was able to get the project to copy.
 
 Note that when the files are separately dragged in they can produce
 different results.
 
 The paths are valid but but it seems like the copy routines do not
 understand relative and mixed paths.
 
 Now I get
 
 created clean copy of the project in D:\Projects\Game\Game\Game.dustmite
 Failed to make stdin stream inheritable by child process (Access is
 denied.)
 
 so now getting that error about stdin that I get from a fresh project...
 this seems like a bug in Visual D too.
 


This is probably a recently fixed problem in std.process and should be
fine in https://github.com/dlang/visuald/releases/tag/v0.48.0-beta2
Nov 08 2018
prev sibling next sibling parent Pham <home home.com> writes:
On Sunday, 4 November 2018 at 17:16:55 UTC, Michelle Long wrote:

 C:\a\b\b\b.dustmite\..\..\..\..\x\y\pegged\dynamic\grammar.d: 
 The process cannot access the file because it is being used by 
 another process

 When I run dustmite on a project.

 The x is a junction. Nothing is accessing grammar.d except 
 maybe visual studio internally... but even if it were it still 
 should be accessible for copying. (after all I can open the 
 file).

 Seems visual D craps out because it is a junction?

 Could this be remedied?

 I use junctions often because it helps with organization(rather 
 than duplicating).

 It may not be from it being a junction, but if it is not I 
 can't imagine why it is having trouble with it(assuming the 
 error message is correct).


Suggestion, when getting lock error, use Process Explorer to 
inspect which process has an open handle on it and work from 
there.  You can google Process Explorer and down load it from 
Microsoft website
Cheer
Nov 06 2018
prev sibling parent Michelle Long <HappyDance321 gmail.com> writes:
still having problems with this.

My csproj file uses relative paths and backtracks.

..\somepath.

When I replace ..\somepath with the absolute path such as 
`c:\somepath` the project copying works.

We talked about this and I thought things were fixed so I'm not 
sure what  is going on still. I'd like to avoid using absolute 
paths if possible because it makes harder copying the project to 
a new project at a different path if I need to do that.

Seems that .. is simply not handled properly by something.
Dec 14 2018