Go Mod Ultimate Getting Started

Go mod Tutorial

Original link

  Go mod is a officially announced project dependency solution in the Go language. Go mod (formerly known as vgo) were officially released in Go1.11. They are ready for Go1.14 and can be used for production. Go officially encourages all users to migrate from other dependency management tools to Go mod.

  And Go1.14, finally officially released today, Go official call you use:

  So in this article today, I will bring you the *Ultimate Getting Started for Go mod, and welcome everyone to discuss it together.

What is Go mod

  Go mod is a dependency solution for the Go language. It was released in Go1.11, grew up in Go1.12, enriched in Go1.13, and officially recommended for production in Go1.14.

  Go mod is currently integrated in Go’s toolchain. As long as Go is installed, Go moudles can be used naturally, and the emergence of Go mod has also resolved several common controversial issues before Go1.11:

-Go has a long history of dependency management issues. -Retire existing GOPATH usage models. -Other dependency management tools in the unified community (providing migration capabilities).

Those bits of GOPATH

  We mentioned that one of the problems solved by Go mod is to “eliminate” GOPATH, but what is GOPATH? Why did you use GOPATH before Go1.11 and gradually recommend using Go mod after Go1.11? Do you recommend GOPATH again?

What is GOPATH

  Let’s first look at the first question, what is GOPATH, we can enter the following command to view:

$ go env
GOPATH="/Users/eddycjy/go"
...

  After we enter the go env command line, we can see the result of the GOPATH variable. We enter the directory to view it, as follows:

go
├── bin
├── pkg
└── src
    ├── github.com
    ├── golang.org
    ├── google.golang.org
    ├── gopkg.in
    ....

  The GOPATH directory contains three subdirectories, namely:

-bin: Stores the compiled binary files. -pkg: stores pre-compiled object files to speed up subsequent compilation of the program. -src: stores all .go files or source code. When writing Go applications, packages and libraries, they are usually stored in the path $ GOPATH/src/github.com/foo/bar.

  Therefore, when using GOPATH mode, we need to store the application code in a fixed $ GOPATH / src directory, and if you execute go get to pull external dependencies, it will be downloaded and installed automatically in the $ GOPATH directory.

Why deprecate GOPATH mode

  Store the .go file or source code under $ GOPATH / src of GOPATH. We can call it the“ GOPATH ”mode. This mode seems to have no problem, so why do we discard Use it, see the following reasons:

There is no concept of version control in GOPATH mode, and it has fatal flaws, which will cause at least the following problems:

• When executing go get, you cannot convey any expectation of version information, that is, you cannot know which version you are currently updating, nor can you pull the specific version you expect by specifying.
• When running Go applications, you cannot guarantee that other people and the third-party libraries you expect to rely on are the same version, that is to say, in the management of project dependent libraries, you cannot guarantee that all the dependent versions are consistent.
• You can’t deal with different versions of v1, v2, v3, etc., because the import path in GOPATH mode is the same, all are github.com/foo/bar.
• Go language officially promoted Go mod (formerly vgo) from Go1.11. Go1.1 is no longer recommended to use GOPATH. Go mod are also becoming stable, so it is not necessary for new projects to continue to use GOPATH mode.

Product in GOPATH mode

  Go1 was released on March 28, 2012, and Go1.11 was officially released on August 25, 2018 (data source: GitHub Tag). In this gap time, there is no such thing as Go mod. In the early days, it may be okay to say that because it has just been released and is not used by many people, there is no obvious exposure, but there are more and more people using Go language in the later period. What should we do?

  At this time, a large number of dependency solutions gradually emerged in the community, making it difficult to choose, including a model of the vendor directory that we are familiar with, and this type of dependency management, which was once considered an “official” dep. tool.

  But why dep is not becoming official propaganda, it is actually because as Russ Cox and other members of the Go team continue to discuss in depth, it is found that some details of dep seem to be more and more unsuitable for Go, so the official adopted another proposal. The way to advance, the result of the solution was to release vgo (the predecessor of Go mod, you do n’t need to understand it), and it finally evolved into the Go mod we now see. It also officially entered in Go1.11. Go’s toolchain.

  So instead of “products in the GOPATH mode”, history provides important lessons for the current, so Go mod appeared.

Basic use of Go mod

  After a preliminary understanding of the past and present life of Go mod, we officially entered the use of Go mod. First, we will create a Go mod project from scratch (in principle, the directory created should not be placed in GOPATH).

Commands provided

  In Go mod, we can use the following commands:

Provided environment variables

  There are the following common environment variables in Go mod, we can view them with the go env com

$ go env
GO111MODULE="auto"
GOPROXY="https://proxy.golang.org,direct"
GONOPROXY=""
GOSUMDB="sum.golang.org"
GONOSUMDB=""
GOPRIVATE=""
...

GO111MODULE

  Go provides an environment variable GO111MODULE as a switch for Go mod, which allows setting the following parameters:

-auto: Enable Go mod as long as the project includes a go.mod file. Currently it is still the default in Go1.11 to Go1.14. -on: Enable Go mod. The recommended setting will be the default value in future versions. -off: Disable Go mod, not recommended.

GO111MODULE history

  You may notice that the name GO111MODULE is more “strange”. In fact, there are often such staged variables in Go. The name GO111MODULE represents the variables added to Go in 1.11 for Module.

  For example, in the Go 1.5 version, a system environment variable GO15VENDOREXPERIMENT was also released, which is used to enable support for the vendor directory. At that time, the default value was not enabled, and it was only experimental. It then changed the default value to ON in Go1.6 version, and finally became official, and the GO15VENDOREXPERIMENT system variable exited the stage of history.

  In the future, GO111MODULE, a system environment variable, will also face this problem. It will also be adjusted to the default value of on. ), Then remove the support of GO111MODULE, we guess that GO111MODULE should be removed in Go2, because if you remove the support of GO111MODULE directly, there will be compatibility problems.

GOPROXY

  This environment variable is mainly used to set the Go module proxy, and its role is to enable Go to pull the module version from the traditional VCS method and pull directly through the mirror site.

  The default value of GOPROXY is: https://proxy.golang.org,direct, this has a very serious problem, that is, proxy.golang.org is not accessible in China, so this will directly jam you The first step, so you must set the domestic Go module proxy when you start Go mod, and execute the following command:

$ go env -w GOPROXY=https://goproxy.cn,direct

  The value of GOPROXY is a , separated list of Go module agents. It allows multiple module agents to be set. If you do not want to use it, you can also set it to “off”. This will prevent Go from being used in subsequent operations Any Go module proxy.

what is direct

  In the value just set, we can find the “direct” flag in the value list. What does it do?

  In fact, “direct” is a special indicator that instructs Go to go to the source address of the module version to fetch (such as GitHub, etc.). The scenario is as follows: When the last Go module proxy in the value list returns a 404 or 410 error , Go automatically tries the next one in the list. When it encounters “direct”, it goes back to the source, that is, it goes back to the source address to grab. When it encounters EOF, it terminates and throws an error like “invalid version: unknown revision …”

GOSUMDB

  Its value is a Go checksum database, which is used to ensure that the module version data that has been pulled has not been tampered with when pulling module versions (whether from the source site or through the Go module proxy). There may be tampering and it will be suspended immediately.

  The default value of GOSUMDB is: sum.golang.org, which is not accessible in China, but GOSUMDB can be proxied by the Go module proxy (see: Proxying a Checksum Database for details).

  So we can solve it by setting GOPROXY, and the module proxy goproxy.cn we set up previously can support the proxy sum.golang.org, so after setting GOPROXY, you don’t need to care too much.   In addition, if there is a custom requirement for the value of GOSUMDB, it supports the following formats:

• Format 1：<SUMDB_NAME>+<PUBLIC_KEY>。
• Format 1：<SUMDB_NAME>+<PUBLIC_KEY> <SUMDB_URL>。

  It can also be set to “off”, which prevents Go from verifying the module version in subsequent operations.

GONOPROXY/GONOSUMDB/GOPRIVATE

  These three environment variables are used in current projects that rely on private mod, such as your company’s private git repository, or private libraries in github, which are all private mod and must be set. Pull failed.

  In more detail, it is the scenario when you rely on a module that is not accessible by the Go module agent specified by GOPROXY or the Go checksum database specified by GOSUMDB.

  It is generally recommended to set GOPRIVATE directly, and its value will be used as the default value of GONOPROXY and GONOSUMDB, so the best recommended posture is to use GOPRIVATE directly.

  And their values are a module path prefix separated by a comma “,”, that is, you can set multiple, for example:

$ go env -w GOPRIVATE="git.example.com,github.com/eddycjy/mquote"

  Once set, mod prefixed with git.xxx.com and github.com/eddycjy/mquote are considered private mod.

If you don’t want to reset it every time, we can also use wildcards, for example:

$ go env -w GOPRIVATE="*.example.com"

  In this way, all subdomains with a module path of example.com (for example: git.example.com) will not go through the Go module proxy and Go checksum database. Note that example.com itself is not included.

Open Go mod

  At present, Go mod are not enabled by default, so Go provides the environment variable GO111MODULE as a switch for Go mod, which allows the following parameters to be set:

• auto: Enable Go mod as long as the project includes a go.mod file. Currently it is still the default in Go1.11 to Go1.14.
• on: Enable Go mod. The recommended setting will be the default value in future versions.
• off: Disable Go mod, not recommended.

  If you are not sure what your current value is, you can execute the go env command to see the results

$ go env
GO111MODULE="off"
...

  If you need to change the value of GO111MODULE, it is recommended to set it through the go env command:

 $ go env -w GO111MODULE=on

  However, it should be noted that if the corresponding system environment variable has a value (set it), go env does not support overwriting, otherwise the following error message will appear: warning: go env -w GO111MODULE = .. . does not override conflicting OS environment variable.

Or you can achieve this by directly setting the system environment variables (writing the corresponding .bash_profile file):

$ export GO111MODULE=on

Initialize the project

  After starting Go mod, we need to create a sample project for demonstration, execute the following command:

$ mkdir -p $HOME/eddycjy/module-repo
$ cd $HOME/eddycjy/module-repo

  Then initialize Go mod as follows:

$ go mod init github.com/eddycjy/module-repo
go: creating new go.mod: module github.com/eddycjy/module-repo

  When executing the go mod init command, we specified the module import path as github.com/eddycjy/module-repo. Next we create the main.go file in the root directory of the project, as follows:

package main

import (
    "fmt"
    "github.com/eddycjy/mquote"
)

func main() {
	fmt.Println(mquote.GetHello())
}

  Then execute the go get github.com / eddycjy / mquote command in the project root directory, as follows:

$ go get github.com/eddycjy/mquote
go: finding github.com/eddycjy/mquote latest
go: downloading github.com/eddycjy/mquote v0.0.0-20200220041913-e066a990ce6f
go: extracting github.com/eddycjy/mquote v0.0.0-20200220041913-e066a990ce6f

View go.mod file

  When the project is initialized, a go.mod file is generated, which is the most important identifier necessary for the Go mod project to be enabled. It is also the identifier when the GO111MODULE value is auto. It describes the current project (that is, the current module). Meta information, each line starts with a verb.

  After we have just initialized and simply pulled, we look at the go.mod file again, the basic content is as follows:

module github.com/eddycjy/module-repo

go 1.13

require (
	github.com/eddycjy/mquote v0.0.0-20200220041913-e066a990ce6f
)

  For further explanation, we simulate the following:

module github.com/eddycjy/module-repo

go 1.13

require (
    example.com/apple v0.1.2
    example.com/banana v1.2.3
    example.com/banana/v2 v2.3.4
    example.com/pear // indirect
    example.com/strawberry // incompatible
)

exclude example.com/banana v1.2.4
replace example.com/apple v0.1.2 => example.com/fried v0.1.0
replace example.com/banana => example.com/fish

• module: used to define the module path of the current project.
• go: It is used to identify the Go language version of the current module. The value is the version when the module was initialized. At present, it is only an identification function.
• require: used to set a specific module version.
• exclude: Used to exclude a specific module version from use.
• replace: Used to replace one module version with another module version.

  In addition, you will find that there is an indirect logo behind example.com / pear. The indirect logo indicates that the module is an indirect dependency, that is, in the import statement in the current application, there is no explicit reference to this module. It may be that you first manually pull it down with go get, or it may depend on the mod you depend on. There are several situations.

View go.sum file

  After pulling the module dependencies for the first time, you will find that there is an extra go.sum file that lists all the module versions that the current project directly or indirectly depends on, and indicates the SHA-256 hash value of those module versions In the future, Go will ensure that the module versions that the project depends on will not be tampered with.

github.com/eddycjy/mquote v0.0.1 h1:4QHXKo7J8a6J/k8UA6CiHhswJQs0sm2foAQQUq8GFHM=
github.com/eddycjy/mquote v0.0.1/go.mod h1:ZtlkDs7Mriynl7wsDQ4cU23okEtVYqHwl7F1eDh4qPg=
github.com/eddycjy/mquote/module/tour v0.0.1 h1:cc+pgV0LnR8Fhou0zNHughT7IbSnLvfUZ+X3fvshrv8=
github.com/eddycjy/mquote/module/tour v0.0.1/go.mod h1:8uL1FOiQJZ4/1hzqQ5mv4Sm7nJcwYu41F3nZmkiWx5I=
...

  We can see that a module path may have the following two types:

github.com/eddycjy/mquote v0.0.1 h1:4QHXKo7J8a6J/k8UA6CiHhswJQs0sm2foAQQUq8GFHM=
github.com/eddycjy/mquote v0.0.1/go.mod h1:ZtlkDs7Mriynl7wsDQ4cU23okEtVYqHwl7F1eDh4qPg=

  The h1 hash is the Go module that unpacks the zip file of the target module version, and then hashes all the files in the package in order, and then composes their hash results into a total hash value according to a fixed format and algorithm.

  The h1 hash and the go.mod hash exist either at the same time or only go.mod hash. Under what circumstances will there be no h1 hash, that is, when Go thinks that a module version is definitely not needed, it will omit its h1 hash, and there will be no h1 hash, only go.mod hash.

View global cache

  We just successfully pulled the github.com/eddycjy/mquote module, and the results of the pull are cached in the $GOPATH/pkg/mod and $GOPATH/pkg/sumdb directories, and in the mod directory It will be stored in the format github.com/foo/bar, as follows:

mod
├── cache
├── github.com
├── golang.org
├── google.golang.org
├── gopkg.in
...

  It should be noted that only one copy of the data of the same module version is cached, and all other mod are shared and used. If you want to clean up all cached module version data, you can execute the go clean -modcache command.

Go get behavior under Go mod

  When pulling project dependencies, you will find that the pulling process is divided into three major steps, namely, finding (finding), downloading (downloading), and extracting (extracting), and the pulling information is divided into three sections :

  It should be noted that the commit time of the pulled version is based on the UTC time zone, not the local time zone. At the same time, we will find that the version pulled by our go get command is v0.0.0, because we directly execute The version obtained by go get -u does not specify any version information, and it is selected by Go mod according to internal rules.

go get pull behavior

  Just now we used the go get command to pull the new dependencies, then what functions does go get provide, the common pull commands are as follows:

  Then I want to choose how to implement the specific version, as follows:

Go get version selection

  Let’s review the go get github.com/eddycjy/mquote we pulled, and the result is v0.0.0-20200220041913-e066a990ce6f. Looking at the behavior of go get mentioned above, you may still There will be some doubts, what is the version selection rule of go get without specifying any version, that is why go get pulls v0.0.0, when is it Will pull tags with normal version number. Actually this needs to distinguish between two cases, as follows:

• The pulled mod have release tags:
  • If there is only a single module, then take the tag with the largest major version number.
  • If there are multiple mod, the corresponding module path is calculated, and the tag with the largest major version number is used (the module path of the submodule tag will have a prefix requirement)
• The module being pulled has not been published tags:
  • By default, the commithash of the latest commit of the master branch is taken.
  • No tags have been posted

No published tags

  So why does it pull v0.0.0, because github.com / eddycjy / mquote does not publish any tags, as follows:

  Therefore, it takes the commit time and commithash of the latest commit of the master branch by default, which is 20200220041913-e066a990ce6f, which belongs to the second case.

Have tags

  When the project has published tags, there are also multiple modes, that is, there is only a single module and multiple mod. We unified the display with multiple mod, because in the case of multiple mod, a single module is already included. Used, as shown below:

  In this project, we have added two tags: v0.0.1 and module / tour / v0.0.1. At this time you may be wondering, why is it so strange to tag module / tour / v0.0.1, does this have any purpose?

  In fact, it is the tag expression of multiple mod of Go mod in the same project. Its main directory structure is:

mquote
├── go.mod
├── module
│   └── tour
│       ├── go.mod
│       └── tour.go
└── quote.go

  You can see that there is a go.mod file in the root directory of the mquote project, and a go.mod file in the module / tour directory. The corresponding relationship between module import and version information is as follows:

Import main and submod

  In combination with the above, if you pull the main module, you still execute the following command as usual:

$ go get github.com/eddycjy/[email protected]
go: finding github.com/eddycjy/mquote v0.0.1
go: downloading github.com/eddycjy/mquote v0.0.1
go: extracting github.com/eddycjy/mquote v0.0.1

  If you want to pull the submodule, execute the following command:

$ go get github.com/eddycjy/mquote/module/[email protected]
go: finding github.com/eddycjy/mquote/module v0.0.1
go: finding github.com/eddycjy/mquote/module/tour v0.0.1
go: downloading github.com/eddycjy/mquote/module/tour v0.0.1
go: extracting github.com/eddycjy/mquote/module/tour v0.0.1

  We compare the pull of the main module and the submodule, and you will find that the pull of the submodule will be an extra step. It will first find github.com / eddycjy / mquote / module, then continue to calculate, and finally pull to module / tour.

Go mod import path description

Import paths for different versions

  In the previous module pull and reference, you will find that our module import paths are github.com/eddycjy/mquote and github.com/eddycjy/mquote/module/tour, it seems that there is nothing special.   Actually not. In fact, Go mod omit the version number when the main version number is v0 and v1, and the main version number needs to be explicitly specified if the main version number is v2 or above, otherwise there will be conflicts, and its tags and mod The approximate correspondence of the import path is as follows:

  In short, when the major version numbers are v0 and v1, you do not need to include the major version information in the module import path. After the v1 version, that is, v2, you must add the major version number at the end of the module import path. , When quoting, it needs to be adjusted to the following format:

import (
    "github.com/eddycjy/mquote/v2/example"
)

  Also ignoring major version numbers v0 and v1 is mandatory (not optional), so each package has only one clear and canonical import path.

Why ignore major version numbers for v0 and v1

-1. The reason for ignoring the v1 version in the import path is: considering that many developers create packages that will never change once they reach the v1 version, this is officially encouraged, and it is not considered that all these developers unintentionally release v2 Should be forced to have a clear v1 version suffix, which will cause the v1 version to become “noisy” and meaningless.

-2. The reason for ignoring the v0 version in the import path is that according to the semantic version specification, these versions of v0 have no compatibility guarantee at all. The need for an explicit v0 version of the identity does not help much to ensure compatibility.

Semantic versioning for Go mod

  We keep referring to the version number in the use of Go mod, which is essentially called the “semantic version”, assuming our version number is v1.2.3, as follows:

  Its version format is “major version number. Minor version number. Revision number”.

• Major version number: when you make incompatible API changes.
• Minor version number: Added functionality when you made backwards compatible.
• Revision number: when you made backwards compatibility fixes.

  Assume that you are the previous version number or special case, you can append the version information to the “major version number. Minor version number. Revision number” as an extension, as follows:

  So far, we have introduced the two types of version number methods supported by Go mod. When we release a new version to tag, we need to pay attention to it. Otherwise, version numbers that do not follow the semantic version rules cannot be pulled.

Go mod minimal version selection

  Now we already have a module and a published tag, but a module often depends on many other mod, and different mod will likely rely on different versions of the same module when they depend on it, as shown below (from Russ Cox):

  In the above dependencies, module A depends on module B and module C, while module B depends on module D, module C depends on mod D and F, module D depends on module E, and different versions of the same module also depend on the corresponding Different versions of the module. So how does Go mod choose the version at this time, and which version is chosen?

  According to the proposal, we can learn that Go mod will sort out the list of dependent versions of each module, and finally get a build list, as shown below (from Russ Cox):

  We see rough list and final list. The difference between the two is the module D (v1.3, v1.4) that is repeatedly referenced. The final list uses the v1.4 version of module D. The main reasons are:

• Semantic version control: Because the v1.3 and v1.4 changes of module D are minor version number changes, under the constraints of the semantic version, v1.4 must be backward compatible with v1.3 , So it ’s considered non-destructive and compatible.

• Specification of the module import path: the major version number is different, the module import path is different, so if there is an incompatibility, the major version number will change, and the module import path will naturally change, so it will not be the same as the first Conflicting foundations.

Go.sum file to submit

  In theory, both go.mod and go.sum files should be submitted to your Git repository.

  Suppose we do not upload the go.sum file, it will cause everyone to execute Go mod related commands, and a new go.sum will be generated, that is, it will be pulled to the upstream again. It may be tampered when pulling Yes, there will be great security risks, and the verification content with the baseline version (the first person submitted, the expected version) is lost, so the go.sum file needs to be submitted.

Summarize

  So far we have introduced the past and present life of Go mod, the basic use and the behavior conversion of the go get command in Go mod mode. At the same time, we have performed common multi-version import paths, semantic version control, and minimum version selection rules for multiple mod Gave a rough introduction.

  The growth and development of Go mod has gone through a certain process. If you are a new reader, you can start a project based on Go mod directly. If you have an old project, then it is time to consider switching over. Go1.14 is ready Ready and recommended for you to use.

Seference

• wiki/mod
• wiki/vgo
• Full of Go mod and goproxy.cn