Permalink to this article – https://ift.tt/ifDAGMP
Since setting up our internal private Go module proxy at the company last year, our private proxy has worked pretty well. It stands to reason that this sequel should not exist :).
As the days passed, the Go team gradually grew, and the air was filled with the “scent of Go”.
Suddenly one day, the business line considers replacing the currently used gerrit with gitlab . The reason for using gerrit in the first place is unknown, but my guess is to use gerrit’s powerful and unique code review mechanism and corresponding workflow. However, due to the rapid changes in business requirements, each iteration has many functions, and the “+2” review mechanism will be useless later.
If the gerrit review workflow is not used, what value does gerrit have? From the feedback from the administrator, gerrit is also more complicated to configure, especially permissions. The superposition of the two has the idea of migrating to gitlab. One of the things in front of the Go team is how to adapt our internal private go module proxy to gitlab .
If you are still not clear about the principle of building a private Go module proxy, please read the “Private Go Module Pulling Scheme in Small Factory” before reading further.
Adapt to gitlab
Review the schematic for our private Go module proxy:
Based on this schematic diagram, we came to the conclusion after analysis: to adapt to the gitlab warehouse, it is actually very simple, just modify the real repo address of each module in the configuration file of govanityurls, which is also in line with replacing a back-end code warehouse The principle of service theory is that developers have no sense of it.
Next, we create a foo repo on gitlab, and its corresponding module path is mycompany.com/go/foo. We use ssh to pull the gitlab repo, first add the public key of the host where goproxy is located to the gitlab ssh key. Then fill in the clone address given in the prompt box of the gitlab clone button: [email protected]:go/foo.git into the vanity.yaml file:
//vanity.yaml ... ... /go/foo: repo: ssh://[email protected]:go/foo.git vcs: git
I created a test program on a development machine, the program imports mycompany.com/go/foo, and the result of executing the go mod tidy command is as follows:
$go mod tidy go: finding module for package mycompany.com/go/foo demo imports mycompany.com/go/foo: cannot find module providing package mycompany.com/go/foo: module mycompany.com/go/foo: reading http://10.10.20.20:10000/mycompany.com/go/foo/@v/list: 404 Not Found server response: go list -m -json -versions mycompany.com/go/foo@latest: go: mycompany.com/go/foo@latest: unrecognized import path "mycompany.com/go/foo": http://mycompany.com/go/foo?go-get=1: invalid repo root "ssh://[email protected]:go/foo.git": parse "ssh://[email protected]:go/foo.git": invalid port ":go" after host
Judging from the response content returned by goproxy, it seems that the go command used by goproxy cannot be recognized: “ssh://[email protected]:go/foo.git”, it is believed that the semicolon after 10.10.30.30 should be followed by a port , instead of go.
We replace the repo with the following format:
/go/foo: repo: ssh://[email protected]:80/go/foo.git vcs: gitRestart govanityurls and re-execute go mod tidy, but still get an error:
$go mod tidy go: finding module for package mycompany.com/go/foo demo imports mycompany.com/go/foo: cannot find module providing package mycompany.com/go/foo: module mycompany.com/go/foo: reading http://10.10.20.20:10000/mycompany.com/go/foo/@v/list: 404 Not Found server response: go list -m -json -versions mycompany.com/go/foo@latest: go: module mycompany.com/go/foo: git ls-remote -q origin in /root/.bin/goproxycache/pkg/mod/cache/vcs/4d37c02c151342112bd2d7e6cf9c0508b31b8fe1cf27063da6774aa0f53d872f: exit status 128: kex_exchange_identification: Connection closed by remote host fatal: Could not read from remote repository.
It is also an error to git clone [email protected]:80/go/foo.git directly on the host! ssh does not work, let’s try the http method again. Using the http method, you need to enter the username and password every time you clone, which is not suitable for goproxy. It’s time to put personal tokens into action! Assign the personal token on gitlab, and then create ~/.netrc locally as follows:
# cat ~/.netrc machine 10.10.30.30 login tonybai password [your personal token]
Then we changed the repo in vanity.yaml to the following form:
// vanity.yaml /go/foo: repo: http://10.10.30.30/go/foo.git vcs: git
In this way, execute go mod tidy again, and the foo repository will be pulled down smoothly.
Q&A
1. git clone error
When building goproxy, we usually manually verify whether the private repository can be successfully pulled through git on the goproxy server. If the following error message appears in git clone, what is the problem?
$ git clone ssh://[email protected]:29418/go/common Cloning into 'common'... Unable to negotiate with 10.10.30.30 port 29418: no matching key exchange method found. Their offer: diffie-hellman-group14-sha1,diffie-hellman-group1-sha1 fatal: Could not read from remote repository. Please make sure you have the correct access rights and the repository exists.
The error message here is actually very clear. The git server supports two key exchange methods, diffie-hellman-group1-sha1 and diffie-hellman-group14-sha1, but the git client does not support either of them by default.
How to solve it? We need to add a configuration .ssh/config to the host where goproxy is located:
// ~/.ssh/config Host 10.10.30.30 HostName 10.10.30.30 User tonybai Port 29418 KexAlgorithms +diffie-hellman-group1-sha1 IdentityFile ~/.ssh/id_rsa
With this configuration, we can clone successfully.
2. Use a non-secure connection
Some children’s shoes will encounter the following problems after using this solution:
$go get mycompany.com/go/common@latest go: module mycompany.com/go/common: reading http://10.10.30.30:10000/mycompany.com/go/common/@v/list: 404 Not Found server response: go list -m -json -versions mycompany.com/go/common@latest: go list -m: mycompany.com/go/common@latest: unrecognized import path "mycompany.com/go/common": https fetch: Get "https://mycompany.com/go/common?go-get=1": dial tcp 127.0.0.1:443: connect: connection refused
First, the server response information obtained by go get indicates that 127.0.0.1:443 cannot be connected. Check the nginx access.log of the goproxy host, and there is no log. Indicates that goproxy did not initiate a request. That is to say, the problem lies in the go list command. Why does it want to connect to 127.0.0.1:443? Our code server uses http instead of https.
This reminds me of GOINSECURE added in Go 1.14. The go command uses the secure method by default, that is, https to access the code repository. If it is not required to obtain the module through https, or even if https is used, the server certificate is no longer verified, then the GOINSECURE environment variable needs to be set, for example;
export GOINSECURE="mycompany.com"
In this way, when you get the go module under mycompany.com/…, the above error will not appear!
“Gopher Tribe” Knowledge Planet aims to create a high-quality Go learning and advanced community! High-quality first published Go technical articles, “three-day” first published reading rights, analysis of the current situation of Go language development twice a year, read the fresh Gopher daily 1 hour in advance every day, online courses, technical columns, book content preview, must answer within six hours Guaranteed to meet all your needs about the Go language ecosystem! In 2022, the Gopher tribe will be fully revised, and will continue to share knowledge, skills and practices in the Go language and Go application fields, and add many forms of interaction. Everyone is welcome to join!
I love texting : Enterprise-level SMS platform customization development expert https://51smspush.com/. smspush : A customized SMS platform that can be deployed within the enterprise, with three-network coverage, not afraid of large concurrent access, and can be customized and expanded; the content of the SMS is determined by you, no longer bound, with rich interfaces, long SMS support, and optional signature. On April 8, 2020, China’s three major telecom operators jointly released the “5G Message White Paper”, and the 51 SMS platform will also be newly upgraded to the “51 Commercial Message Platform” to fully support 5G RCS messages.
The famous cloud hosting service provider DigitalOcean released the latest hosting plan. The entry-level Droplet configuration is upgraded to: 1 core CPU, 1G memory, 25G high-speed SSD, and the price is 5$/month. Friends who need to use DigitalOcean can open this link : https://ift.tt/mYOWVTS to open your DO host road.
Gopher Daily Archive Repository – https://ift.tt/gDCLrO8
my contact information:
- Weibo: https://ift.tt/x1bOZHj
- Blog: tonybai.com
- github: https://ift.tt/rCv4gGZ
Business cooperation methods: writing, publishing books, training, online courses, partnership entrepreneurship, consulting, advertising cooperation.
© 2022, bigwhite . All rights reserved.
This article is reprinted from https://tonybai.com/2022/06/18/the-approach-to-go-get-private-go-module-in-house-part2/
This site is for inclusion only, and the copyright belongs to the original author.