<@UB89GSWCF> hi, having a discussion with Erik on ...
# atmos-dev
a
@Andriy Knysh (Cloud Posse) hi, having a discussion with Erik on go-getter v1 vs v2. Would appreciate if you share your view, see below.
a
@Andrey Kalmykov i'll review this
did you estimate how much effort it would take to upgrade to v2? and how many breaking changes we will have?
a
@Andriy Knysh (Cloud Posse) thanks, no, haven't made any estimations yet, that's more an MVP of how that actually works. I only tested in on vendor pull and managed to build a working example. The bright side seems is that now we can download stuff over http not only git clone. Wanted more to check with you and Erik if this is a go/no go.
e
I think it will depend on LOE
Are you absolutely certain there's no way to do it in v1
a
@Erik Osterman (Cloud Posse) thanks, my current understanding is that with git clone approach of v1 we can not use a gitub token as it isn't an http request (it is a git clone).
a
maybe we can combine v1 and v2 for now, detect the scheme and call one or the other (w/o making too many changes to the current code)
e
Ya, I was thinking something along those lines as well.
my current understanding is that with git clone approach of v1
My observation is that it downloads a tarball archive and decompresses it, and that it's not actually doing a git clone
and that that download is over https
a
@Erik Osterman (Cloud Posse) i might be wrong but I think it does do git clone. Download with v2 over https works lot faster. So do you propose to do a check of incoming scheme and if it ha https - call v2, and if does not - v1, for backward compatibility?
e
Ok, I think you're right
It converts https urls to
git::
What if instead we just write a custom detector fornow.
Copy code
package main

import (
	"fmt"
	"net/url"
	"strings"

	"github.com/hashicorp/go-getter"
)

// Custom GitHub detector
func CustomGitHubDetector(u *url.URL) (string, map[string]string, error) {
	if u.Host != "github.com" {
		return "", nil, nil // Not a GitHub URL, skip this detector
	}

	parts := strings.SplitN(u.Path, "/", 4)
	if len(parts) < 3 {
		return "", nil, fmt.Errorf("invalid GitHub URL %q", u)
	}

	// Extract GitHub token from the environment
	githubToken := os.Getenv("GITHUB_TOKEN")
	if githubToken != "" {
		// Add the token to the URL
		u.User = url.User(githubToken)
	}

	// Convert to a git:: URL
	if len(parts) == 4 {
		return "git::" + u.String(), map[string]string{}, nil
	}

	return "git::" + fmt.Sprintf("https://%s/%s/%s.git", u.Host, parts[1], parts[2]), map[string]string{}, nil
}
Copy code
// Register custom detectors
	getter.Detectors = []getter.Detector{
		// Add the custom GitHub detector first
		getter.DetectorFunc(CustomGitHubDetector),
		// Add other default detectors as fallback
		getter.DetectGitHub,  // Original GitHub detector (optional)
		getter.DetectGzip,    // Detect .gz files
		getter.DetectGit,     // Detect git:: URLs
		getter.DetectHTTP,    // Detect HTTP(S) URLs
	}

	// Use the getter.Client with the modified detectors
	client := &getter.Client{
		Src:  "https://github.com/user/repo.git",
		Dst:  "./repo",
		Mode: getter.ClientModeAny,
	}

	if err := client.Get(); err != nil {
		fmt.Printf("Error: %s\n", err)
	}
a
Actually i tried doing
git::https://<token>@github.com/...
and it still does git clone not using github token at all, or does not if repo is private. If the token doesn’t have the exact right Git credentials/scopes, git clone won’t accept it. So the v1 would not use token.
e
or does not if repo is private.
Do you mean then it doesn't work with a private repo even when you have a hardcoded token (just for testing)?
a
@Erik Osterman (Cloud Posse) Hi, Just checked again in the morning and yes, it git:https//<token>@github.com/... does allow me to download from my private repo. So, the solution for now is to stay on v1 and implement custom detector? Im going to rollback my latest commit with upgrade then and do this instead.
this 1
e
Let’s do that for now, to reduce the scope
👍 1