www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Sourceforge projects

reply Vitaly Livshic <vitaly_himki mail.ru> writes:
Hi, everyone.

I develop small library for D language and attempt to publish it 
on code.dlang.org. Originally, sourceforge project was created. 
However, when I tried to publish it site show error's message:

Please enter a valid project URL or tuple to a GitHub, GitLab or 
BitBucket project, such as https://gitlab.com/dlang/dub or 
bitbucket/dlang/dub.

Restriction on a specific sites and version control is unexpected 
for open-source project. Is there are any possibility to use 
sourceforge or non-git repositories?
Oct 14 2020
parent reply rikki cattermole <rikki cattermole.co.nz> writes:
On 15/10/2020 2:18 AM, Vitaly Livshic wrote:

 Restriction on a specific sites and version control is unexpected for 
 open-source project. Is there are any possibility to use sourceforge or 
 non-git repositories?


It can be completely expected the current behavior.

Version control systems like git don't provide a standard way via a web 
server to obtain a compressed archive at a specific commit/tag/branch.

Nor does it provide access to meta-data. Like tags or branches.

You can get this information and archive by checking out the repository, 
which is more complex to perform and requires the VCS to be installed on 
the node that does it.

Here are the source code for the VCS suppliers that dub-registry 
supports[0].

Note: I would recommend heavily to not create new projects on SF, they 
have been known in recent years to do some nasty stuff with any binaries 
that you may upload.

[0] 
https://github.com/dlang/dub-registry/tree/master/source/dubregistry/repositories
Oct 14 2020
next sibling parent starcanopy <starcanopy protonmail.com> writes:
On Wednesday, 14 October 2020 at 16:41:26 UTC, rikki cattermole 
wrote:

 Note: I would recommend heavily to not create new projects on 
 SF, they have been known in recent years to do some nasty stuff 
 with any binaries that you may upload.


I don't host code on it, but I do download stuff from it. Since 
SF changed hands a year ago, there hasn't been any saliently 
shady stuff. I feel bad for the new owners, as this well-deserved 
notoriety will be difficult to absolve.
Oct 14 2020
prev sibling parent reply norm <normical gmail.com> writes:
On Wednesday, 14 October 2020 at 16:41:26 UTC, rikki cattermole 
wrote:

 On 15/10/2020 2:18 AM, Vitaly Livshic wrote:

 Restriction on a specific sites and version control is 
 unexpected for open-source project. Is there are any 
 possibility to use sourceforge or non-git repositories?


 It can be completely expected the current behavior.

 Version control systems like git don't provide a standard way 
 via a web server to obtain a compressed archive at a specific 
 commit/tag/branch.

 Nor does it provide access to meta-data. Like tags or branches.

 You can get this information and archive by checking out the 
 repository, which is more complex to perform and requires the 
 VCS to be installed on the node that does it.

 Here are the source code for the VCS suppliers that 
 dub-registry supports[0].

 Note: I would recommend heavily to not create new projects on 
 SF, they have been known in recent years to do some nasty stuff 
 with any binaries that you may upload.

 [0] 
 https://github.com/dlang/dub-registry/tree/master/source/dubregistry/repositories



This sounds like an arbitrary limitation in dub tbh and nothing 
to do with git. The standard git commands work with SF hosted 
repos just like they do with github, gitlab and bitbucket.

Git archive will get you a compressed archive if all you want are 
the files, but why does dub want to do this?

Git can clone the metadata without checking out HEAD if that is 
all dub requires, which sounds more like what the dub registry 
requires.
Oct 14 2020
next sibling parent reply Vitaly Livshic <vitaly_himki mail.ru> writes:
On Wednesday, 14 October 2020 at 22:21:21 UTC, norm wrote:

 Git can clone the metadata without checking out HEAD if that is 
 all dub requires, which sounds more like what the dub registry 
 requires.


There are several tens public repositories for open-source 
projects. If SourceForge has bad reputation, project can be 
hosted by any of these sites. Why limit by three commercial ones 
for open source code?

Git is most popular system, but Mercurial, Fossil, Darcs have 
users too. Allowed sites have no support.
Oct 14 2020
parent norm <norm.rowtree gmail.com> writes:
On Thursday, 15 October 2020 at 03:33:10 UTC, Vitaly Livshic 
wrote:

 On Wednesday, 14 October 2020 at 22:21:21 UTC, norm wrote:

 Git can clone the metadata without checking out HEAD if that 
 is all dub requires, which sounds more like what the dub 
 registry requires.


 There are several tens public repositories for open-source 
 projects. If SourceForge has bad reputation, project can be 
 hosted by any of these sites. Why limit by three commercial 
 ones for open source code?

 Git is most popular system, but Mercurial, Fossil, Darcs have 
 users too. Allowed sites have no support.


I don't think dub needs to support anything but git for now, even 
bitbucket has dropped support for mercurial.

But I agree dub registry should be able to support *any* valid 
url that points at any open git repo.
Oct 14 2020
prev sibling next sibling parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On 10/14/20 6:21 PM, norm wrote:

 On Wednesday, 14 October 2020 at 16:41:26 UTC, rikki cattermole wrote:

 On 15/10/2020 2:18 AM, Vitaly Livshic wrote:

 Restriction on a specific sites and version control is unexpected for 
 open-source project. Is there are any possibility to use sourceforge 
 or non-git repositories?


 It can be completely expected the current behavior.

 Version control systems like git don't provide a standard way via a 
 web server to obtain a compressed archive at a specific 
 commit/tag/branch.

 Nor does it provide access to meta-data. Like tags or branches.

 You can get this information and archive by checking out the 
 repository, which is more complex to perform and requires the VCS to 
 be installed on the node that does it.

 Here are the source code for the VCS suppliers that dub-registry 
 supports[0].

 Note: I would recommend heavily to not create new projects on SF, they 
 have been known in recent years to do some nasty stuff with any 
 binaries that you may upload.

 [0] 
 https://github.com/dlang/dub-registry/tree/master/source/dubr
gistry/repositories 


 
 
 This sounds like an arbitrary limitation in dub tbh and nothing to do 
 with git. The standard git commands work with SF hosted repos just like 
 they do with github, gitlab and bitbucket.


Dub does not use git commands to introspect what versions exist, etc. It 
uses web APIs that are provided by the hosting systems. It's just the 
case that dub doesn't have support for those other systems. If you 
submit a PR to add sourceforge support, I'm sure it would be accepted.

Here are the objects which deal with the different repository types:
https://github.com/dlang/dub-registry/tree/master/source/dubregistry/repositories


 Git archive will get you a compressed archive if all you want are the 
 files, but why does dub want to do this?
 
 Git can clone the metadata without checking out HEAD if that is all dub 
 requires, which sounds more like what the dub registry requires.


I'm sure also, improvements to dub that allow fetching the info through 
a git interface would be accepted as well.

-Steve
Oct 15 2020
parent reply rikki cattermole <rikki cattermole.co.nz> writes:
On 16/10/2020 2:05 AM, Steven Schveighoffer wrote:

 I'm sure also, improvements to dub that allow fetching the info through 
 a git interface would be accepted as well.


This isn't guaranteed to work (without checking out the repository).

It depends on how git is hosted as well as git on the server and client 
is configured.
Oct 15 2020
next sibling parent Steven Schveighoffer <schveiguy gmail.com> writes:
On 10/15/20 11:14 AM, rikki cattermole wrote:

 On 16/10/2020 2:05 AM, Steven Schveighoffer wrote:

 I'm sure also, improvements to dub that allow fetching the info 
 through a git interface would be accepted as well.


 
 This isn't guaranteed to work (without checking out the repository).
 
 It depends on how git is hosted as well as git on the server and client 
 is configured.


Sure, that would be a discussion on said PR. My point was, it's possible 
to address the suggestion "Git can clone the metadata without checking 
out HEAD". Suggest the mechanism, and get a response.

In a rough guess, I tried git clone -n, but it still downloads all the 
data of a repository.

-Steve
Oct 15 2020
prev sibling parent reply norm <normical gmail.com> writes:
On Thursday, 15 October 2020 at 15:14:15 UTC, rikki cattermole 
wrote:

 On 16/10/2020 2:05 AM, Steven Schveighoffer wrote:

 I'm sure also, improvements to dub that allow fetching the 
 info through a git interface would be accepted as well.


 This isn't guaranteed to work (without checking out the 
 repository).

 It depends on how git is hosted as well as git on the server 
 and client is configured.


My apologies I stand corrected, I thought git had the interface 
to read just all the metadata without pulling all the data.

I still think it is an arbitrary limitation of dub not to support 
a git interface but I get that it would require someone to put in 
some serious time on the project to make it happen.
Oct 15 2020
parent reply rikki cattermole <rikki cattermole.co.nz> writes:
On 16/10/2020 12:00 PM, norm wrote:

 My apologies I stand corrected, I thought git had the interface to read 
 just all the metadata without pulling all the data.


When I last looked at it a few years ago, my statement was true.

After more git versions being released, it may no longer be true for a 
lot of use cases.

Either way, you still need the archive as well (which is a separate 
feature that is also optional, or was at one point). So you may end up 
with needing to clone it fully anyway.

I don't see a problem with checking out a repository to get the info + 
archive unless it is a big repo. So that would end up being our fallback 
anyway if you want non-supported providers.
Oct 15 2020
parent sarn <sarn theartofmachinery.com> writes:
On Thursday, 15 October 2020 at 23:26:03 UTC, rikki cattermole 
wrote:

 On 16/10/2020 12:00 PM, norm wrote:

 My apologies I stand corrected, I thought git had the 
 interface to read just all the metadata without pulling all 
 the data.


 When I last looked at it a few years ago, my statement was true.

 After more git versions being released, it may no longer be 
 true for a lot of use cases.

 Either way, you still need the archive as well (which is a 
 separate feature that is also optional, or was at one point). 
 So you may end up with needing to clone it fully anyway.

 I don't see a problem with checking out a repository to get the 
 info + archive unless it is a big repo. So that would end up 
 being our fallback anyway if you want non-supported providers.


It's not the same as an API that just gets you the tags, but 
partial cloning and sparse checkouts are supported by recent git 
versions (could also throw in shallow clones for repos with a lot 
of history).  Maybe that's good enough.

https://www.git-scm.com/docs/partial-clone
https://github.blog/2020-01-17-bring-your-monorepo-down-to-size-with-sparse-checkout/

Here's a quick test with vibe.d (full clone is 24.84MB).  Most 
projects have under a few hundred KB of history.

$ git clone --filter=tree:0 --no-checkout 
https://github.com/vibe-d/vibe.d
Cloning into 'vibe.d'...
remote: Enumerating objects: 12, done.
remote: Counting objects: 100% (12/12), done.
remote: Compressing objects: 100% (12/12), done.
remote: Total 6232 (delta 0), reused 9 (delta 0), pack-reused 6220
Receiving objects: 100% (6232/6232), 1.56 MiB | 1.43 MiB/s, done.
Resolving deltas: 100% (64/64), done.
$ cd vibe.d/
$ git tag | head
0.7.20-rc.1
test
test2
v0.7.0
v0.7.0-beta.3
v0.7.1
v0.7.10
v0.7.11
v0.7.12
v0.7.13
$ git log | head
commit b5afadbe53849db5f3beacbbac1f2ec028f24770
Merge: a567566 61bff81
Author: Sönke Ludwig <sludwig rejectedsoftware.com>
Date:   Thu Oct 15 12:44:53 2020 +0200



     Fix refcounted FD leak in HTTPClient

commit 61bff81cfbf97843fdd1a47390bf5f20fba5b876
$
Oct 18 2020
prev sibling parent Jacob Carlborg <doob me.com> writes:
On 2020-10-15 00:21, norm wrote:


 Git can clone the metadata without checking out HEAD if that is all dub 
 requires, which sounds more like what the dub registry requires.


With the DMD release 2.094.0, Dub now seems to support arbitrary git 
repositories [1].

[1] https://dlang.org/changelog/2.094.0.html#git-paths

-- 
/Jacob Carlborg
Oct 16 2020