Add self-contained gometalinter build tooling.

This commit is contained in:
Will Rouesnel
2017-06-06 21:39:41 +10:00
parent 0de0311c22
commit e2b6c973a1
710 changed files with 277204 additions and 35 deletions

View File

@@ -0,0 +1,56 @@
### Please only report errors with gometalinter itself
gometalinter relies on underlying linters to detect issues in source code.
If your issue seems to be related to an underlying linter, please report an
issue against that linter rather than gometalinter. For a full list of linters
and their repositories please see the [README](README.md).
### Do you want to upgrade a vendored linter?
Please send a PR. We use [GVT](https://github.com/FiloSottile/gvt). It should be as simple as:
```
go get github.com/FiloSottile/gvt
cd _linters
gvt update <linter>
git add <paths>
```
### Before you report an issue
Sometimes gometalinter will not report issues that you think it should. There
are three things to try in that case:
#### 1. Update to the latest build of gometalinter and all linters
go get -u github.com/alecthomas/gometalinter
gometalinter --install
If you're lucky, this will fix the problem.
#### 2. Analyse the debug output
If that doesn't help, the problem may be elsewhere (in no particular order):
1. Upstream linter has changed its output or semantics.
2. gometalinter is not invoking the tool correctly.
3. gometalinter regular expression matches are not correct for a linter.
4. Linter is exceeding the deadline.
To find out what's going on run in debug mode:
gometalinter --debug
This will show all output from the linters and should indicate why it is
failing.
#### 3. Run linters manually
The output of `gometalinter --debug` should show the exact commands gometalinter
is running. Run these commands from the command line to determine if the linter
or gometaliner is at fault.
#### 4. Report an issue.
Failing all else, if the problem looks like a bug please file an issue and
include the output of `gometalinter --debug`

View File

@@ -0,0 +1,19 @@
Copyright (C) 2012 Alec Thomas
Permission is hereby granted, free of charge, to any person obtaining a copy of
this software and associated documentation files (the "Software"), to deal in
the Software without restriction, including without limitation the rights to
use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies
of the Software, and to permit persons to whom the Software is furnished to do
so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.

View File

@@ -0,0 +1,317 @@
# Go Meta Linter
[![Build Status](https://travis-ci.org/alecthomas/gometalinter.png)](https://travis-ci.org/alecthomas/gometalinter) [![Gitter chat](https://badges.gitter.im/alecthomas.png)](https://gitter.im/alecthomas/Lobby)
<!-- MarkdownTOC -->
- [Editor integration](#editor-integration)
- [Supported linters](#supported-linters)
- [Configuration file](#configuration-file)
- [Installing](#installing)
- [Comment directives](#comment-directives)
- [Quickstart](#quickstart)
- [FAQ](#faq)
- [Exit status](#exit-status)
- [What's the best way to use `gometalinter` in CI?](#whats-the-best-way-to-use-gometalinter-in-ci)
- [How do I make `gometalinter` work with Go 1.5 vendoring?](#how-do-i-make-gometalinter-work-with-go-15-vendoring)
- [Why does `gometalinter --install` install a fork of gocyclo?](#why-does-gometalinter---install-install-a-fork-of-gocyclo)
- [Gometalinter is not working](#gometalinter-is-not-working)
- [1. Update to the latest build of gometalinter and all linters](#1-update-to-the-latest-build-of-gometalinter-and-all-linters)
- [2. Analyse the debug output](#2-analyse-the-debug-output)
- [3. Report an issue.](#3-report-an-issue)
- [Details](#details)
- [Checkstyle XML format](#checkstyle-xml-format)
<!-- /MarkdownTOC -->
The number of tools for statically checking Go source for errors and warnings
is impressive.
This is a tool that concurrently runs a whole bunch of those linters and
normalises their output to a standard format:
<file>:<line>:[<column>]: <message> (<linter>)
eg.
stutter.go:9::warning: unused global variable unusedGlobal (varcheck)
stutter.go:12:6:warning: exported type MyStruct should have comment or be unexported (golint)
It is intended for use with editor/IDE integration.
## Editor integration
- [SublimeLinter plugin](https://github.com/alecthomas/SublimeLinter-contrib-gometalinter).
- [Atom go-plus package](https://atom.io/packages/go-plus).
- [Emacs Flycheck checker](https://github.com/favadi/flycheck-gometalinter).
- [Go for Visual Studio Code](https://marketplace.visualstudio.com/items?itemName=lukehoban.Go).
- Vim/Neovim
- [Neomake](https://github.com/neomake/neomake).
- [Syntastic](https://github.com/scrooloose/syntastic/wiki/Go:---gometalinter) `let g:syntastic_go_checkers = ['gometalinter']`.
- [ale](https://github.com/w0rp/ale) `let g:ale_linters = {'go': ['gometalinter']}`
- [vim-go](https://github.com/fatih/vim-go) with the `:GoMetaLinter` command.
## Supported linters
- [go vet](https://golang.org/cmd/vet/) - Reports potential errors that otherwise compile.
- [go vet --shadow](https://golang.org/cmd/vet/#hdr-Shadowed_variables) - Reports variables that may have been unintentionally shadowed.
- [gotype](https://golang.org/x/tools/cmd/gotype) - Syntactic and semantic analysis similar to the Go compiler.
- [deadcode](https://github.com/tsenart/deadcode) - Finds unused code.
- [gocyclo](https://github.com/alecthomas/gocyclo) - Computes the cyclomatic complexity of functions.
- [golint](https://github.com/golang/lint) - Google's (mostly stylistic) linter.
- [varcheck](https://github.com/opennota/check) - Find unused global variables and constants.
- [structcheck](https://github.com/opennota/check) - Find unused struct fields.
- [aligncheck](https://github.com/opennota/check) - Warn about un-optimally aligned structures.
- [errcheck](https://github.com/kisielk/errcheck) - Check that error return values are used.
- [dupl](https://github.com/mibk/dupl) - Reports potentially duplicated code.
- [ineffassign](https://github.com/gordonklaus/ineffassign/blob/master/list) - Detect when assignments to *existing* variables are not used.
- [interfacer](https://github.com/mvdan/interfacer) - Suggest narrower interfaces that can be used.
- [unconvert](https://github.com/mdempsky/unconvert) - Detect redundant type conversions.
- [goconst](https://github.com/jgautheron/goconst) - Finds repeated strings that could be replaced by a constant.
- [gosimple](https://github.com/dominikh/go-tools/tree/master/cmd/gosimple) - Report simplifications in code.
- [staticcheck](https://github.com/dominikh/go-tools/tree/master/cmd/staticcheck) - Statically detect bugs, both obvious and subtle ones.
- [gas](https://github.com/GoASTScanner/gas) - Inspects source code for security problems by scanning the Go AST.
Disabled by default (enable with `--enable=<linter>`):
- [testify](https://github.com/stretchr/testify) - Show location of failed testify assertions.
- [test](http://golang.org/pkg/testing/) - Show location of test failures from the stdlib testing module.
- [gofmt -s](https://golang.org/cmd/gofmt/) - Checks if the code is properly formatted and could not be further simplified.
- [goimports](https://godoc.org/golang.org/x/tools/cmd/goimports) - Checks missing or unreferenced package imports.
- [lll](https://github.com/walle/lll) - Report long lines (see `--line-length=N`).
- [misspell](https://github.com/client9/misspell) - Finds commonly misspelled English words.
- [unparam](https://github.com/mvdan/unparam) - Find unused function parameters.
- [unused](https://github.com/dominikh/go-tools/tree/master/cmd/unused) - Find unused variables.
- [safesql](https://github.com/stripe/safesql) - Finds potential SQL injection vulnerabilities.
Additional linters can be added through the command line with `--linter=NAME:COMMAND:PATTERN` (see [below](#details)).
## Configuration file
gometalinter now supports a JSON configuration file which can be loaded via
`--config=<file>`. The format of this file is determined by the Config struct
in `config.go`.
The configuration file mostly corresponds to command-line flags, with the following exceptions:
- Linters defined in the configuration file will overlay existing definitions, not replace them.
- "Enable" defines the exact set of linters that will be enabled.
Here is an example configuration file:
```json
{
"DisableAll": true,
"Enable": ["deadcode", "unconvert"]
}
```
## Installing
There are two options for installing gometalinter.
1. Install a stable version, eg. `go get -u gopkg.in/alecthomas/gometalinter.v1`.
I will generally only tag a new stable version when it has passed the Travis
regression tests. The downside is that the binary will be called `gometalinter.v1`.
2. Install from HEAD with: `go get -u github.com/alecthomas/gometalinter`.
This has the downside that changes to gometalinter may break.
## Comment directives
gometalinter supports suppression of linter messages via comment directives. The
form of the directive is:
```
// nolint[: <linter>[, <linter>, ...]]
```
Suppression works in the following way:
1. Line-level suppression
A comment directive suppresses any linter messages on that line.
eg. In this example any messages for `a := 10` will be suppressed and errcheck
messages for `defer r.Close()` will also be suppressed.
```go
a := 10 // nolint
a = 2
defer r.Close() // nolint: errcheck
```
2. Statement-level suppression
A comment directive at the same indentation level as a statement it
immediately precedes will also suppress any linter messages in that entire
statement.
eg. In this example all messages for `SomeFunc()` will be suppressed.
```go
// nolint
func SomeFunc() {
}
```
Implementation details: gometalinter now performs parsing of Go source code,
to extract linter directives and associate them with line ranges. To avoid
unnecessary processing, parsing is on-demand: the first time a linter emits a
message for a file, that file is parsed for directives.
## Quickstart
Install gometalinter (see above).
Install all known linters:
```
$ gometalinter --install
Installing:
structcheck
aligncheck
deadcode
gocyclo
ineffassign
dupl
golint
gotype
goimports
errcheck
varcheck
interfacer
goconst
gosimple
staticcheck
unparam
unused
misspell
lll
gas
safesql
```
Run it:
```
$ cd example
$ gometalinter ./...
stutter.go:13::warning: unused struct field MyStruct.Unused (structcheck)
stutter.go:9::warning: unused global variable unusedGlobal (varcheck)
stutter.go:12:6:warning: exported type MyStruct should have comment or be unexported (golint)
stutter.go:16:6:warning: exported type PublicUndocumented should have comment or be unexported (golint)
stutter.go:8:1:warning: unusedGlobal is unused (deadcode)
stutter.go:12:1:warning: MyStruct is unused (deadcode)
stutter.go:16:1:warning: PublicUndocumented is unused (deadcode)
stutter.go:20:1:warning: duplicateDefer is unused (deadcode)
stutter.go:21:15:warning: error return value not checked (defer a.Close()) (errcheck)
stutter.go:22:15:warning: error return value not checked (defer a.Close()) (errcheck)
stutter.go:27:6:warning: error return value not checked (doit() // test for errcheck) (errcheck)
stutter.go:29::error: unreachable code (vet)
stutter.go:26::error: missing argument for Printf("%d"): format reads arg 1, have only 0 args (vet)
```
Gometalinter also supports the commonly seen `<path>/...` recursive path
format. Note that this can be *very* slow, and you may need to increase the linter `--deadline` to allow linters to complete.
## FAQ
### Exit status
gometalinter sets two bits of the exit status to indicate different issues:
| Bit | Meaning
|-----|----------
| 0 | A linter generated an issue.
| 1 | An underlying error occurred; eg. a linter failed to execute. In this situation a warning will also be displayed.
eg. linter only = 1, underlying only = 2, linter + underlying = 3
### What's the best way to use `gometalinter` in CI?
There are two main problems running in a CI:
1. <s>Linters break, causing `gometalinter --install --update` to error</s> (this is no longer an issue as all linters are vendored).
2. `gometalinter` adds a new linter.
I have solved 1 by vendoring the linters.
For 2, the best option is to disable all linters, then explicitly enable the
ones you want:
gometalinter --disable-all --enable=errcheck --enable=vet --enable=vetshadow ...
### How do I make `gometalinter` work with Go 1.5 vendoring?
`gometalinter` has a `--vendor` flag that just sets `GO15VENDOREXPERIMENT=1`, however the
underlying tools must support it. Ensure that all of the linters are up to date and built with Go 1.5
(`gometalinter --install --force`) then run `gometalinter --vendor .`. That should be it.
### Why does `gometalinter --install` install a fork of gocyclo?
I forked `gocyclo` because the upstream behaviour is to recursively check all
subdirectories even when just a single directory is specified. This made it
unusably slow when vendoring. The recursive behaviour can be achieved with
gometalinter by explicitly specifying `<path>/...`. There is a
[pull request](https://github.com/fzipp/gocyclo/pull/1) open.
### Gometalinter is not working
That's more of a statement than a question, but okay.
Sometimes gometalinter will not report issues that you think it should. There
are three things to try in that case:
#### 1. Update to the latest build of gometalinter and all linters
go get -u github.com/alecthomas/gometalinter
gometalinter --install
If you're lucky, this will fix the problem.
#### 2. Analyse the debug output
If that doesn't help, the problem may be elsewhere (in no particular order):
1. Upstream linter has changed its output or semantics.
2. gometalinter is not invoking the tool correctly.
3. gometalinter regular expression matches are not correct for a linter.
4. Linter is exceeding the deadline.
To find out what's going on run in debug mode:
gometalinter --debug
This will show all output from the linters and should indicate why it is
failing.
#### 3. Report an issue.
Failing all else, if the problem looks like a bug please file an issue and
include the output of `gometalinter --debug`.
## Details
Additional linters can be configured via the command line:
```
$ gometalinter --linter='vet:go tool vet -printfuncs=Infof,Debugf,Warningf,Errorf {path}:PATH:LINE:MESSAGE' .
stutter.go:21:15:warning: error return value not checked (defer a.Close()) (errcheck)
stutter.go:22:15:warning: error return value not checked (defer a.Close()) (errcheck)
stutter.go:27:6:warning: error return value not checked (doit() // test for errcheck) (errcheck)
stutter.go:9::warning: unused global variable unusedGlobal (varcheck)
stutter.go:13::warning: unused struct field MyStruct.Unused (structcheck)
stutter.go:12:6:warning: exported type MyStruct should have comment or be unexported (golint)
stutter.go:16:6:warning: exported type PublicUndocumented should have comment or be unexported (deadcode)
```
## Checkstyle XML format
`gometalinter` supports [checkstyle](http://checkstyle.sourceforge.net/)
compatible XML output format. It is triggered with `--checkstyle` flag:
gometalinter --checkstyle
Checkstyle format can be used to integrate gometalinter with Jenkins CI with the
help of [Checkstyle Plugin](https://wiki.jenkins-ci.org/display/JENKINS/Checkstyle+Plugin).

View File

@@ -0,0 +1,50 @@
package main
import (
"sort"
"strings"
)
type (
issueKey struct {
path string
line, col int
message string
}
multiIssue struct {
*Issue
linterNames []string
}
)
func aggregateIssues(issues chan *Issue) chan *Issue {
out := make(chan *Issue, 1000000)
issueMap := make(map[issueKey]*multiIssue)
go func() {
for issue := range issues {
key := issueKey{
path: issue.Path,
line: issue.Line,
col: issue.Col,
message: issue.Message,
}
if existing, ok := issueMap[key]; ok {
existing.linterNames = append(existing.linterNames, issue.Linter.Name)
} else {
issueMap[key] = &multiIssue{
Issue: issue,
linterNames: []string{issue.Linter.Name},
}
}
}
for _, multi := range issueMap {
issue := multi.Issue
sort.Strings(multi.linterNames)
issue.Linter.Name = strings.Join(multi.linterNames, ", ")
out <- issue
}
close(out)
}()
return out
}

View File

@@ -0,0 +1,66 @@
package main
import (
"encoding/xml"
"fmt"
"gopkg.in/alecthomas/kingpin.v3-unstable"
)
type checkstyleOutput struct {
XMLName xml.Name `xml:"checkstyle"`
Version string `xml:"version,attr"`
Files []*checkstyleFile `xml:"file"`
}
type checkstyleFile struct {
Name string `xml:"name,attr"`
Errors []*checkstyleError `xml:"error"`
}
type checkstyleError struct {
Column int `xml:"column,attr"`
Line int `xml:"line,attr"`
Message string `xml:"message,attr"`
Severity string `xml:"severity,attr"`
Source string `xml:"source,attr"`
}
func outputToCheckstyle(issues chan *Issue) int {
var lastFile *checkstyleFile
out := checkstyleOutput{
Version: "5.0",
}
status := 0
for issue := range issues {
if lastFile != nil && lastFile.Name != issue.Path {
out.Files = append(out.Files, lastFile)
lastFile = nil
}
if lastFile == nil {
lastFile = &checkstyleFile{
Name: issue.Path,
}
}
if config.Errors && issue.Severity != Error {
continue
}
lastFile.Errors = append(lastFile.Errors, &checkstyleError{
Column: issue.Col,
Line: issue.Line,
Message: issue.Message,
Severity: string(issue.Severity),
Source: issue.Linter.Name,
})
status = 1
}
if lastFile != nil {
out.Files = append(out.Files, lastFile)
}
d, err := xml.Marshal(&out)
kingpin.FatalIfError(err, "")
fmt.Printf("%s%s\n", xml.Header, d)
return status
}

View File

@@ -0,0 +1,188 @@
package main
import (
"runtime"
"text/template"
"time"
"gopkg.in/alecthomas/kingpin.v3-unstable"
)
// Config for gometalinter. This can be loaded from a JSON file with --config.
type Config struct { // nolint: aligncheck
// A map of linter name to "<command>:<pattern>".
//
// <command> should always include {path} as the target directory to execute. Globs in <command>
// are expanded by gometalinter (not by the shell).
Linters map[string]string
// The set of linters that should be enabled.
Enable []string
Disable []string
// A map of linter name to message that is displayed. This is useful when linters display text
// that is useful only in isolation, such as errcheck which just reports the construct.
MessageOverride map[string]string
Severity map[string]string
VendoredLinters bool
Format string
Fast bool
Install bool
Update bool
Force bool
DownloadOnly bool
Debug bool
Concurrency int
Exclude []string
Include []string
Skip []string
Vendor bool
Cyclo int
LineLength int
MinConfidence float64
MinOccurrences int
MinConstLength int
DuplThreshold int
Sort []string
Test bool
Deadline time.Duration `json:"-"`
Errors bool
JSON bool
Checkstyle bool
EnableGC bool
Aggregate bool
DeadlineJSONCrutch string `json:"Deadline"`
}
// Configuration defaults.
var (
vetRe = `^(?:vet:.*?\.go:\s+(?P<path>.*?\.go):(?P<line>\d+):(?P<col>\d+):\s*(?P<message>.*))|(?:(?P<path>.*?\.go):(?P<line>\d+):\s*(?P<message>.*))$`
predefinedPatterns = map[string]string{
"PATH:LINE:COL:MESSAGE": `^(?P<path>.*?\.go):(?P<line>\d+):(?P<col>\d+):\s*(?P<message>.*)$`,
"PATH:LINE:MESSAGE": `^(?P<path>.*?\.go):(?P<line>\d+):\s*(?P<message>.*)$`,
}
formatTemplate = &template.Template{}
installMap = map[string]string{
"aligncheck": "github.com/opennota/check/cmd/aligncheck",
"deadcode": "github.com/tsenart/deadcode",
"dupl": "github.com/mibk/dupl",
"errcheck": "github.com/kisielk/errcheck",
"gas": "github.com/GoASTScanner/gas",
"goconst": "github.com/jgautheron/goconst/cmd/goconst",
"gocyclo": "github.com/alecthomas/gocyclo",
"goimports": "golang.org/x/tools/cmd/goimports",
"golint": "github.com/golang/lint/golint",
"gosimple": "honnef.co/go/tools/cmd/gosimple",
"gotype": "golang.org/x/tools/cmd/gotype",
"ineffassign": "github.com/gordonklaus/ineffassign",
"interfacer": "github.com/mvdan/interfacer/cmd/interfacer",
"lll": "github.com/walle/lll/cmd/lll",
"misspell": "github.com/client9/misspell/cmd/misspell",
"safesql": "github.com/stripe/safesql",
"staticcheck": "honnef.co/go/tools/cmd/staticcheck",
"structcheck": "github.com/opennota/check/cmd/structcheck",
"unconvert": "github.com/mdempsky/unconvert",
"unparam": "github.com/mvdan/unparam",
"unused": "honnef.co/go/tools/cmd/unused",
"varcheck": "github.com/opennota/check/cmd/varcheck",
}
acceptsEllipsis = map[string]bool{
"aligncheck": true,
"errcheck": true,
"golint": true,
"gosimple": true,
"interfacer": true,
"staticcheck": true,
"structcheck": true,
"test": true,
"varcheck": true,
"unconvert": true,
}
slowLinters = []string{"structcheck", "varcheck", "errcheck", "aligncheck", "testify", "test", "interfacer", "unconvert", "deadcode", "safesql", "staticcheck", "unparam", "unused", "gosimple"}
sortKeys = []string{"none", "path", "line", "column", "severity", "message", "linter"}
// Linter definitions.
linterDefinitions = map[string]string{
"aligncheck": `aligncheck {path}:^(?:[^:]+: )?(?P<path>.*?\.go):(?P<line>\d+):(?P<col>\d+):\s*(?P<message>.+)$`,
"deadcode": `deadcode {path}:^deadcode: (?P<path>.*?\.go):(?P<line>\d+):(?P<col>\d+):\s*(?P<message>.*)$`,
"dupl": `dupl -plumbing -threshold {duplthreshold} {path}/*.go:^(?P<path>.*?\.go):(?P<line>\d+)-\d+:\s*(?P<message>.*)$`,
"errcheck": `errcheck -abspath {path}:PATH:LINE:COL:MESSAGE`,
"gas": `gas -fmt=csv {path}/*.go:^(?P<path>.*?\.go),(?P<line>\d+),(?P<message>[^,]+,[^,]+,[^,]+)`,
"goconst": `goconst -min-occurrences {min_occurrences} -min-length {min_const_length} {path}:PATH:LINE:COL:MESSAGE`,
"gocyclo": `gocyclo -over {mincyclo} {path}:^(?P<cyclo>\d+)\s+\S+\s(?P<function>\S+)\s+(?P<path>.*?\.go):(?P<line>\d+):(\d+)$`,
"gofmt": `gofmt -l -s {path}/*.go:^(?P<path>.*?\.go)$`,
"goimports": `goimports -l {path}/*.go:^(?P<path>.*?\.go)$`,
"golint": "golint -min_confidence {min_confidence} {path}:PATH:LINE:COL:MESSAGE",
"gosimple": "gosimple {path}:PATH:LINE:COL:MESSAGE",
"gotype": "gotype -e {tests=-a} {path}:PATH:LINE:COL:MESSAGE",
"ineffassign": `ineffassign -n {path}:PATH:LINE:COL:MESSAGE`,
"interfacer": `interfacer {path}:PATH:LINE:COL:MESSAGE`,
"lll": `lll -g -l {maxlinelength} {path}/*.go:PATH:LINE:MESSAGE`,
"misspell": "misspell -j 1 {path}/*.go:PATH:LINE:COL:MESSAGE",
"safesql": `safesql {path}:^- (?P<path>.*?\.go):(?P<line>\d+):(?P<col>\d+)$`,
"staticcheck": "staticcheck {path}:PATH:LINE:COL:MESSAGE",
"structcheck": `structcheck {tests=-t} {path}:^(?:[^:]+: )?(?P<path>.*?\.go):(?P<line>\d+):(?P<col>\d+):\s*(?P<message>.+)$`,
"test": `go test {path}:^--- FAIL: .*$\s+(?P<path>.*?\.go):(?P<line>\d+): (?P<message>.*)$`,
"testify": `go test {path}:Location:\s+(?P<path>.*?\.go):(?P<line>\d+)$\s+Error:\s+(?P<message>[^\n]+)`,
"unconvert": "unconvert {path}:PATH:LINE:COL:MESSAGE",
"unparam": `unparam {path}:PATH:LINE:COL:MESSAGE`,
"unused": `unused {path}:PATH:LINE:COL:MESSAGE`,
"varcheck": `varcheck {path}:^(?:[^:]+: )?(?P<path>.*?\.go):(?P<line>\d+):(?P<col>\d+):\s*(?P<message>.*)$`,
"vet": `go tool vet {path}/*.go:` + vetRe,
"vetshadow": `go tool vet --shadow {path}/*.go:` + vetRe,
}
pathsArg = kingpin.Arg("path", "Directories to lint. Defaults to \".\". <path>/... will recurse.").Strings()
config = &Config{
Format: "{{.Path}}:{{.Line}}:{{if .Col}}{{.Col}}{{end}}:{{.Severity}}: {{.Message}} ({{.Linter}})",
Severity: map[string]string{
"gotype": "error",
"test": "error",
"testify": "error",
"vet": "error",
},
MessageOverride: map[string]string{
"errcheck": "error return value not checked ({message})",
"gocyclo": "cyclomatic complexity {cyclo} of function {function}() is high (> {mincyclo})",
"gofmt": "file is not gofmted with -s",
"goimports": "file is not goimported",
"safesql": "potentially unsafe SQL statement",
"structcheck": "unused struct field {message}",
"unparam": "parameter {message}",
"varcheck": "unused variable or constant {message}",
},
Enable: []string{
"aligncheck",
"deadcode",
"errcheck",
"gas",
"goconst",
"gocyclo",
"golint",
"gosimple",
"gotype",
"ineffassign",
"interfacer",
"staticcheck",
"structcheck",
"unconvert",
"varcheck",
"vet",
"vetshadow",
},
VendoredLinters: true,
Concurrency: runtime.NumCPU(),
Cyclo: 10,
LineLength: 80,
MinConfidence: 0.8,
MinOccurrences: 3,
MinConstLength: 3,
DuplThreshold: 50,
Sort: []string{"none"},
Deadline: time.Second * 30,
}
)

View File

@@ -0,0 +1,164 @@
package main
import (
"go/ast"
"go/parser"
"go/token"
"sort"
"strings"
"sync"
"time"
)
type ignoredRange struct {
col int
start, end int
linters []string
}
func (i *ignoredRange) matches(issue *Issue) bool {
if issue.Line < i.start || issue.Line > i.end {
return false
}
if len(i.linters) == 0 {
return true
}
for _, l := range i.linters {
if l == issue.Linter.Name {
return true
}
}
return false
}
func (i *ignoredRange) near(col, start, end int) bool {
return col == i.col && i.end == start-1
}
type ignoredRanges []*ignoredRange
func (ir ignoredRanges) Len() int { return len(ir) }
func (ir ignoredRanges) Swap(i, j int) { ir[i], ir[j] = ir[j], ir[i] }
func (ir ignoredRanges) Less(i, j int) bool { return ir[i].end < ir[j].end }
type directiveParser struct {
paths []string
lock sync.Mutex
files map[string]ignoredRanges
fset *token.FileSet
}
func newDirectiveParser(paths []string) *directiveParser {
return &directiveParser{
paths: paths,
files: map[string]ignoredRanges{},
fset: token.NewFileSet(),
}
}
// IsIgnored returns true if the given linter issue is ignored by a linter directive.
func (d *directiveParser) IsIgnored(issue *Issue) bool {
d.lock.Lock()
ranges, ok := d.files[issue.Path]
if !ok {
ranges = d.parseFile(issue.Path)
sort.Sort(ranges)
d.files[issue.Path] = ranges
}
d.lock.Unlock()
for _, r := range ranges {
if r.matches(issue) {
return true
}
}
return false
}
// Takes a set of ignoredRanges, determines if they immediately precede a statement
// construct, and expands the range to include that construct. Why? So you can
// precede a function or struct with //nolint
type rangeExpander struct {
fset *token.FileSet
ranges ignoredRanges
}
func (a *rangeExpander) Visit(node ast.Node) ast.Visitor {
if node == nil {
return a
}
startPos := a.fset.Position(node.Pos())
start := startPos.Line
end := a.fset.Position(node.End()).Line
found := sort.Search(len(a.ranges), func(i int) bool {
return a.ranges[i].end+1 >= start
})
if found < len(a.ranges) && a.ranges[found].near(startPos.Column, start, end) {
r := a.ranges[found]
if r.start > start {
r.start = start
}
if r.end < end {
r.end = end
}
}
return a
}
func (d *directiveParser) parseFile(path string) ignoredRanges {
start := time.Now()
debug("nolint: parsing %s for directives", path)
file, err := parser.ParseFile(d.fset, path, nil, parser.ParseComments)
if err != nil {
debug("nolint: failed to parse %q: %s", path, err)
return nil
}
ranges := extractCommentGroupRange(d.fset, file.Comments...)
visitor := &rangeExpander{fset: d.fset, ranges: ranges}
ast.Walk(visitor, file)
debug("nolint: parsing %s took %s", path, time.Since(start))
return visitor.ranges
}
func extractCommentGroupRange(fset *token.FileSet, comments ...*ast.CommentGroup) (ranges ignoredRanges) {
for _, g := range comments {
for _, c := range g.List {
text := strings.TrimLeft(c.Text, "/ ")
var linters []string
if strings.HasPrefix(text, "nolint") {
if strings.HasPrefix(text, "nolint:") {
for _, linter := range strings.Split(text[7:], ",") {
linters = append(linters, strings.TrimSpace(linter))
}
}
pos := fset.Position(g.Pos())
rng := &ignoredRange{
col: pos.Column,
start: pos.Line,
end: fset.Position(g.End()).Line,
linters: linters,
}
ranges = append(ranges, rng)
}
}
}
return
}
func (d *directiveParser) in(n ast.Node, issue *Issue) bool {
start := d.fset.Position(n.Pos())
end := d.fset.Position(n.End())
return issue.Line >= start.Line && issue.Line <= end.Line
}
func filterIssuesViaDirectives(directives *directiveParser, issues chan *Issue) chan *Issue {
out := make(chan *Issue, 1000000)
go func() {
for issue := range issues {
if !directives.IsIgnored(issue) {
out <- issue
}
}
close(out)
}()
return out
}

908
tools/vendor/github.com/alecthomas/gometalinter/main.go generated vendored Normal file
View File

@@ -0,0 +1,908 @@
package main
import (
"bytes"
"encoding/json"
"fmt"
"os"
"os/exec"
"os/user"
"path/filepath"
"reflect"
"regexp"
"runtime"
"sort"
"strconv"
"strings"
"sync"
"text/template"
"time"
"github.com/google/shlex"
"gopkg.in/alecthomas/kingpin.v3-unstable"
)
// Severity of linter message.
type Severity string
// Linter message severity levels.
const ( // nolint
Warning Severity = "warning"
Error Severity = "error"
)
var (
// Locations to look for vendored linters.
vendoredSearchPaths = [][]string{
{"github.com", "alecthomas", "gometalinter", "_linters"},
{"gopkg.in", "alecthomas", "gometalinter.v1", "_linters"},
}
)
type Linter struct {
Name string `json:"name"`
Command string `json:"command"`
CompositeCommand string `json:"composite_command,omitempty"`
Pattern string `json:"pattern"`
InstallFrom string `json:"install_from"`
SeverityOverride Severity `json:"severity,omitempty"`
MessageOverride string `json:"message_override,omitempty"`
regex *regexp.Regexp
}
func (l *Linter) MarshalJSON() ([]byte, error) {
return json.Marshal(l.Name)
}
func (l *Linter) String() string {
return l.Name
}
func LinterFromName(name string) *Linter {
s := linterDefinitions[name]
parts := strings.SplitN(s, ":", 2)
if len(parts) < 2 {
kingpin.Fatalf("invalid linter: %q", name)
}
pattern := parts[1]
if p, ok := predefinedPatterns[pattern]; ok {
pattern = p
}
re, err := regexp.Compile("(?m:" + pattern + ")")
kingpin.FatalIfError(err, "invalid regex for %q", name)
return &Linter{
Name: name,
Command: s[0:strings.Index(s, ":")],
Pattern: pattern,
InstallFrom: installMap[name],
SeverityOverride: Severity(config.Severity[name]),
MessageOverride: config.MessageOverride[name],
regex: re,
}
}
type sortedIssues struct {
issues []*Issue
order []string
}
func (s *sortedIssues) Len() int { return len(s.issues) }
func (s *sortedIssues) Swap(i, j int) { s.issues[i], s.issues[j] = s.issues[j], s.issues[i] }
// nolint: gocyclo
func (s *sortedIssues) Less(i, j int) bool {
l, r := s.issues[i], s.issues[j]
for _, key := range s.order {
switch key {
case "path":
if l.Path > r.Path {
return false
}
case "line":
if l.Line > r.Line {
return false
}
case "column":
if l.Col > r.Col {
return false
}
case "severity":
if l.Severity > r.Severity {
return false
}
case "message":
if l.Message > r.Message {
return false
}
case "linter":
if l.Linter.Name > r.Linter.Name {
return false
}
}
}
return true
}
func init() {
kingpin.Flag("config", "Load JSON configuration from file.").Action(loadConfig).String()
kingpin.Flag("disable", "Disable previously enabled linters.").PlaceHolder("LINTER").Short('D').Action(disableAction).Strings()
kingpin.Flag("enable", "Enable previously disabled linters.").PlaceHolder("LINTER").Short('E').Action(enableAction).Strings()
kingpin.Flag("linter", "Define a linter.").PlaceHolder("NAME:COMMAND:PATTERN").StringMapVar(&config.Linters)
kingpin.Flag("message-overrides", "Override message from linter. {message} will be expanded to the original message.").PlaceHolder("LINTER:MESSAGE").StringMapVar(&config.MessageOverride)
kingpin.Flag("severity", "Map of linter severities.").PlaceHolder("LINTER:SEVERITY").StringMapVar(&config.Severity)
kingpin.Flag("disable-all", "Disable all linters.").Action(disableAllAction).Bool()
kingpin.Flag("enable-all", "Enable all linters.").Action(enableAllAction).Bool()
kingpin.Flag("format", "Output format.").PlaceHolder(config.Format).StringVar(&config.Format)
kingpin.Flag("vendored-linters", "Use vendored linters (recommended).").BoolVar(&config.VendoredLinters)
kingpin.Flag("fast", "Only run fast linters.").BoolVar(&config.Fast)
kingpin.Flag("install", "Attempt to install all known linters.").Short('i').BoolVar(&config.Install)
kingpin.Flag("update", "Pass -u to go tool when installing.").Short('u').BoolVar(&config.Update)
kingpin.Flag("force", "Pass -f to go tool when installing.").Short('f').BoolVar(&config.Force)
kingpin.Flag("download-only", "Pass -d to go tool when installing.").BoolVar(&config.DownloadOnly)
kingpin.Flag("debug", "Display messages for failed linters, etc.").Short('d').BoolVar(&config.Debug)
kingpin.Flag("concurrency", "Number of concurrent linters to run.").PlaceHolder(fmt.Sprintf("%d", runtime.NumCPU())).Short('j').IntVar(&config.Concurrency)
kingpin.Flag("exclude", "Exclude messages matching these regular expressions.").Short('e').PlaceHolder("REGEXP").StringsVar(&config.Exclude)
kingpin.Flag("include", "Include messages matching these regular expressions.").Short('I').PlaceHolder("REGEXP").StringsVar(&config.Include)
kingpin.Flag("skip", "Skip directories with this name when expanding '...'.").Short('s').PlaceHolder("DIR...").StringsVar(&config.Skip)
kingpin.Flag("vendor", "Enable vendoring support (skips 'vendor' directories and sets GO15VENDOREXPERIMENT=1).").BoolVar(&config.Vendor)
kingpin.Flag("cyclo-over", "Report functions with cyclomatic complexity over N (using gocyclo).").PlaceHolder("10").IntVar(&config.Cyclo)
kingpin.Flag("line-length", "Report lines longer than N (using lll).").PlaceHolder("80").IntVar(&config.LineLength)
kingpin.Flag("min-confidence", "Minimum confidence interval to pass to golint.").PlaceHolder(".80").FloatVar(&config.MinConfidence)
kingpin.Flag("min-occurrences", "Minimum occurrences to pass to goconst.").PlaceHolder("3").IntVar(&config.MinOccurrences)
kingpin.Flag("min-const-length", "Minimumum constant length.").PlaceHolder("3").IntVar(&config.MinConstLength)
kingpin.Flag("dupl-threshold", "Minimum token sequence as a clone for dupl.").PlaceHolder("50").IntVar(&config.DuplThreshold)
kingpin.Flag("sort", fmt.Sprintf("Sort output by any of %s.", strings.Join(sortKeys, ", "))).PlaceHolder("none").EnumsVar(&config.Sort, sortKeys...)
kingpin.Flag("tests", "Include test files for linters that support this option").Short('t').BoolVar(&config.Test)
kingpin.Flag("deadline", "Cancel linters if they have not completed within this duration.").PlaceHolder("30s").DurationVar(&config.Deadline)
kingpin.Flag("errors", "Only show errors.").BoolVar(&config.Errors)
kingpin.Flag("json", "Generate structured JSON rather than standard line-based output.").BoolVar(&config.JSON)
kingpin.Flag("checkstyle", "Generate checkstyle XML rather than standard line-based output.").BoolVar(&config.Checkstyle)
kingpin.Flag("enable-gc", "Enable GC for linters (useful on large repositories).").BoolVar(&config.EnableGC)
kingpin.Flag("aggregate", "Aggregate issues reported by several linters.").BoolVar(&config.Aggregate)
kingpin.CommandLine.GetFlag("help").Short('h')
}
func loadConfig(app *kingpin.Application, element *kingpin.ParseElement, ctx *kingpin.ParseContext) error {
r, err := os.Open(*element.Value)
if err != nil {
return err
}
defer r.Close() // nolint: errcheck
err = json.NewDecoder(r).Decode(config)
if err != nil {
return err
}
if config.DeadlineJSONCrutch != "" {
config.Deadline, err = time.ParseDuration(config.DeadlineJSONCrutch)
}
for _, disable := range config.Disable {
for i, enable := range config.Enable {
if enable == disable {
config.Enable = append(config.Enable[:i], config.Enable[i+1:]...)
break
}
}
}
return err
}
func disableAction(app *kingpin.Application, element *kingpin.ParseElement, ctx *kingpin.ParseContext) error {
out := []string{}
for _, linter := range config.Enable {
if linter != *element.Value {
out = append(out, linter)
}
}
config.Enable = out
return nil
}
func enableAction(app *kingpin.Application, element *kingpin.ParseElement, ctx *kingpin.ParseContext) error {
config.Enable = append(config.Enable, *element.Value)
return nil
}
func disableAllAction(app *kingpin.Application, element *kingpin.ParseElement, ctx *kingpin.ParseContext) error {
config.Enable = []string{}
return nil
}
func enableAllAction(app *kingpin.Application, element *kingpin.ParseElement, ctx *kingpin.ParseContext) error {
for linter := range linterDefinitions {
config.Enable = append(config.Enable, linter)
}
return nil
}
type Issue struct {
Linter *Linter `json:"linter"`
Severity Severity `json:"severity"`
Path string `json:"path"`
Line int `json:"line"`
Col int `json:"col"`
Message string `json:"message"`
}
func (i *Issue) String() string {
buf := new(bytes.Buffer)
err := formatTemplate.Execute(buf, i)
kingpin.FatalIfError(err, "Invalid output format")
return buf.String()
}
func debug(format string, args ...interface{}) {
if config.Debug {
fmt.Fprintf(os.Stderr, "DEBUG: "+format+"\n", args...)
}
}
func warning(format string, args ...interface{}) {
fmt.Fprintf(os.Stderr, "WARNING: "+format+"\n", args...)
}
func formatLinters() string {
w := bytes.NewBuffer(nil)
for name := range linterDefinitions {
linter := LinterFromName(name)
install := "(" + linter.InstallFrom + ")"
if install == "()" {
install = ""
}
fmt.Fprintf(w, " %s %s\n %s\n %s\n", name, install, linter.Command, linter.Pattern)
}
return w.String()
}
func formatSeverity() string {
w := bytes.NewBuffer(nil)
for name, severity := range config.Severity {
fmt.Fprintf(w, " %s -> %s\n", name, severity)
}
return w.String()
}
type Vars map[string]string
func (v Vars) Copy() Vars {
out := Vars{}
for k, v := range v {
out[k] = v
}
return out
}
func (v Vars) Replace(s string) string {
for k, v := range v {
prefix := regexp.MustCompile(fmt.Sprintf("{%s=([^}]*)}", k))
if v != "" {
s = prefix.ReplaceAllString(s, "$1")
} else {
s = prefix.ReplaceAllString(s, "")
}
s = strings.Replace(s, fmt.Sprintf("{%s}", k), v, -1)
}
return s
}
func main() {
// Linters are by their very nature, short lived, so disable GC.
// Reduced (user) linting time on kingpin from 0.97s to 0.64s.
kingpin.CommandLine.Help = fmt.Sprintf(`Aggregate and normalise the output of a whole bunch of Go linters.
PlaceHolder linters:
%s
Severity override map (default is "warning"):
%s
`, formatLinters(), formatSeverity())
kingpin.Parse()
configureEnvironment()
if config.Install {
installLinters()
return
}
include, exclude := processConfig(config)
start := time.Now()
paths := expandPaths(*pathsArg, config.Skip)
linters := lintersFromFlags()
status := 0
issues, errch := runLinters(linters, paths, *pathsArg, config.Concurrency, exclude, include)
if config.JSON {
status |= outputToJSON(issues)
} else if config.Checkstyle {
status |= outputToCheckstyle(issues)
} else {
status |= outputToConsole(issues)
}
for err := range errch {
warning("%s", err)
status |= 2
}
elapsed := time.Since(start)
debug("total elapsed time %s", elapsed)
os.Exit(status)
}
// nolint: gocyclo
func processConfig(config *Config) (include *regexp.Regexp, exclude *regexp.Regexp) {
// Move configured linters into linterDefinitions.
for name, definition := range config.Linters {
linterDefinitions[name] = definition
}
tmpl, err := template.New("output").Parse(config.Format)
kingpin.FatalIfError(err, "invalid format %q", config.Format)
formatTemplate = tmpl
if !config.EnableGC {
_ = os.Setenv("GOGC", "off")
}
if config.VendoredLinters && config.Install && config.Update {
warning(`Linters are now vendored by default, --update ignored. The original
behaviour can be re-enabled with --no-vendored-linters.
To request an update for a vendored linter file an issue at:
https://github.com/alecthomas/gometalinter/issues/new
`)
config.Update = false
}
// Force sorting by path if checkstyle mode is selected
// !jsonFlag check is required to handle:
// gometalinter --json --checkstyle --sort=severity
if config.Checkstyle && !config.JSON {
config.Sort = []string{"path"}
}
// PlaceHolder to skipping "vendor" directory if GO15VENDOREXPERIMENT=1 is enabled.
// TODO(alec): This will probably need to be enabled by default at a later time.
if os.Getenv("GO15VENDOREXPERIMENT") == "1" || config.Vendor {
if err := os.Setenv("GO15VENDOREXPERIMENT", "1"); err != nil {
warning("setenv GO15VENDOREXPERIMENT: %s", err)
}
config.Skip = append(config.Skip, "vendor")
config.Vendor = true
}
if len(config.Exclude) > 0 {
exclude = regexp.MustCompile(strings.Join(config.Exclude, "|"))
}
if len(config.Include) > 0 {
include = regexp.MustCompile(strings.Join(config.Include, "|"))
}
runtime.GOMAXPROCS(config.Concurrency)
return include, exclude
}
func outputToConsole(issues chan *Issue) int {
status := 0
for issue := range issues {
if config.Errors && issue.Severity != Error {
continue
}
fmt.Println(issue.String())
status = 1
}
return status
}
func outputToJSON(issues chan *Issue) int {
fmt.Println("[")
status := 0
for issue := range issues {
if config.Errors && issue.Severity != Error {
continue
}
if status != 0 {
fmt.Printf(",\n")
}
d, err := json.Marshal(issue)
kingpin.FatalIfError(err, "")
fmt.Printf(" %s", d)
status = 1
}
fmt.Printf("\n]\n")
return status
}
func runLinters(linters map[string]*Linter, paths, ellipsisPaths []string, concurrency int, exclude *regexp.Regexp, include *regexp.Regexp) (chan *Issue, chan error) {
errch := make(chan error, len(linters)*(len(paths)+len(ellipsisPaths)))
concurrencych := make(chan bool, config.Concurrency)
incomingIssues := make(chan *Issue, 1000000)
directives := newDirectiveParser(paths)
processedIssues := filterIssuesViaDirectives(directives, maybeSortIssues(maybeAggregateIssues(incomingIssues)))
wg := &sync.WaitGroup{}
for _, linter := range linters {
// Recreated in each loop because it is mutated by executeLinter().
vars := Vars{
"duplthreshold": fmt.Sprintf("%d", config.DuplThreshold),
"mincyclo": fmt.Sprintf("%d", config.Cyclo),
"maxlinelength": fmt.Sprintf("%d", config.LineLength),
"min_confidence": fmt.Sprintf("%f", config.MinConfidence),
"min_occurrences": fmt.Sprintf("%d", config.MinOccurrences),
"min_const_length": fmt.Sprintf("%d", config.MinConstLength),
"tests": "",
}
if config.Test {
vars["tests"] = "-t"
}
linterPaths := paths
// Most linters don't exclude vendor paths when recursing, so we don't use ... paths.
if acceptsEllipsis[linter.Name] && !config.Vendor && len(ellipsisPaths) > 0 {
linterPaths = ellipsisPaths
}
for _, path := range linterPaths {
wg.Add(1)
deadline := time.After(config.Deadline)
state := &linterState{
Linter: linter,
issues: incomingIssues,
path: path,
vars: vars.Copy(),
exclude: exclude,
include: include,
deadline: deadline,
}
go func() {
concurrencych <- true
err := executeLinter(state)
if err != nil {
errch <- err
}
<-concurrencych
wg.Done()
}()
}
}
go func() {
wg.Wait()
close(incomingIssues)
close(errch)
}()
return processedIssues, errch
}
// nolint: gocyclo
func expandPaths(paths, skip []string) []string {
if len(paths) == 0 {
paths = []string{"."}
}
skipMap := map[string]bool{}
for _, name := range skip {
skipMap[name] = true
}
dirs := map[string]bool{}
for _, path := range paths {
if strings.HasSuffix(path, "/...") {
root := filepath.Dir(path)
_ = filepath.Walk(root, func(p string, i os.FileInfo, err error) error {
if err != nil {
warning("invalid path %q: %s", p, err)
return err
}
base := filepath.Base(p)
skip := skipMap[base] || skipMap[p] || (strings.ContainsAny(base[0:1], "_.") && base != "." && base != "..")
if i.IsDir() {
if skip {
return filepath.SkipDir
}
} else if !skip && strings.HasSuffix(p, ".go") {
dirs[filepath.Clean(filepath.Dir(p))] = true
}
return nil
})
} else {
dirs[filepath.Clean(path)] = true
}
}
out := make([]string, 0, len(dirs))
for d := range dirs {
out = append(out, d)
}
sort.Strings(out)
for _, d := range out {
debug("linting path %s", d)
}
return out
}
func makeInstallCommand(linters ...string) []string {
cmd := []string{"get"}
if config.VendoredLinters {
cmd = []string{"install"}
} else {
if config.Update {
cmd = append(cmd, "-u")
}
if config.Force {
cmd = append(cmd, "-f")
}
if config.DownloadOnly {
cmd = append(cmd, "-d")
}
}
if config.Debug {
cmd = append(cmd, "-v")
}
cmd = append(cmd, linters...)
return cmd
}
func installLintersWithOneCommand(targets []string) error {
cmd := makeInstallCommand(targets...)
debug("go %s", strings.Join(cmd, " "))
c := exec.Command("go", cmd...) // nolint: gas
c.Stdout = os.Stdout
c.Stderr = os.Stderr
return c.Run()
}
func installLintersIndividually(targets []string) {
failed := []string{}
for _, target := range targets {
cmd := makeInstallCommand(target)
debug("go %s", strings.Join(cmd, " "))
c := exec.Command("go", cmd...) // nolint: gas
c.Stdout = os.Stdout
c.Stderr = os.Stderr
if err := c.Run(); err != nil {
warning("failed to install %s: %s", target, err)
failed = append(failed, target)
}
}
if len(failed) > 0 {
kingpin.Fatalf("failed to install the following linters: %s", strings.Join(failed, ", "))
}
}
func installLinters() {
names := make([]string, 0, len(installMap))
targets := make([]string, 0, len(installMap))
for name, target := range installMap {
names = append(names, name)
targets = append(targets, target)
}
namesStr := strings.Join(names, "\n ")
if config.DownloadOnly {
fmt.Printf("Downloading:\n %s\n", namesStr)
} else {
fmt.Printf("Installing:\n %s\n", namesStr)
}
err := installLintersWithOneCommand(targets)
if err == nil {
return
}
warning("failed to install one or more linters: %s (installing individually)", err)
installLintersIndividually(targets)
}
func maybeAggregateIssues(issues chan *Issue) chan *Issue {
if !config.Aggregate {
return issues
}
return aggregateIssues(issues)
}
func maybeSortIssues(issues chan *Issue) chan *Issue {
if reflect.DeepEqual([]string{"none"}, config.Sort) {
return issues
}
out := make(chan *Issue, 1000000)
sorted := &sortedIssues{
issues: []*Issue{},
order: config.Sort,
}
go func() {
for issue := range issues {
sorted.issues = append(sorted.issues, issue)
}
sort.Sort(sorted)
for _, issue := range sorted.issues {
out <- issue
}
close(out)
}()
return out
}
type linterState struct {
*Linter
path string
issues chan *Issue
vars Vars
exclude *regexp.Regexp
include *regexp.Regexp
deadline <-chan time.Time
}
func (l *linterState) InterpolatedCommand() string {
vars := l.vars.Copy()
if l.ShouldChdir() {
vars["path"] = "."
} else {
vars["path"] = l.path
}
return vars.Replace(l.Command)
}
func (l *linterState) ShouldChdir() bool {
return config.Vendor || !strings.HasSuffix(l.path, "/...") || !strings.Contains(l.Command, "{path}")
}
func parseCommand(dir, command string) (string, []string, error) {
args, err := shlex.Split(command)
if err != nil {
return "", nil, err
}
if len(args) == 0 {
return "", nil, fmt.Errorf("invalid command %q", command)
}
exe, err := exec.LookPath(args[0])
if err != nil {
return "", nil, err
}
out := []string{}
for _, arg := range args[1:] {
if strings.Contains(arg, "*") {
pattern := filepath.Join(dir, arg)
globbed, err := filepath.Glob(pattern)
if err != nil {
return "", nil, err
}
for i, g := range globbed {
if strings.HasPrefix(g, dir+string(filepath.Separator)) {
globbed[i] = g[len(dir)+1:]
}
}
out = append(out, globbed...)
} else {
out = append(out, arg)
}
}
return exe, out, nil
}
func executeLinter(state *linterState) error {
debug("linting with %s: %s (on %s)", state.Name, state.Command, state.path)
start := time.Now()
command := state.InterpolatedCommand()
exe, args, err := parseCommand(state.path, command)
if err != nil {
return err
}
debug("executing %s %q", exe, args)
buf := bytes.NewBuffer(nil)
cmd := exec.Command(exe, args...) // nolint: gas
if state.ShouldChdir() {
cmd.Dir = state.path
}
cmd.Stdout = buf
cmd.Stderr = buf
err = cmd.Start()
if err != nil {
return fmt.Errorf("failed to execute linter %s: %s", command, err)
}
done := make(chan bool)
go func() {
err = cmd.Wait()
done <- true
}()
// Wait for process to complete or deadline to expire.
select {
case <-done:
case <-state.deadline:
err = fmt.Errorf("deadline exceeded by linter %s on %s (try increasing --deadline)",
state.Name, state.path)
kerr := cmd.Process.Kill()
if kerr != nil {
warning("failed to kill %s: %s", state.Name, kerr)
}
return err
}
if err != nil {
debug("warning: %s returned %s", command, err)
}
processOutput(state, buf.Bytes())
elapsed := time.Since(start)
debug("%s linter took %s", state.Name, elapsed)
return nil
}
func (l *linterState) fixPath(path string) string {
lpath := strings.TrimSuffix(l.path, "...")
labspath, _ := filepath.Abs(lpath)
if !l.ShouldChdir() {
path = strings.TrimPrefix(path, lpath)
}
if !filepath.IsAbs(path) {
path, _ = filepath.Abs(filepath.Join(labspath, path))
}
if strings.HasPrefix(path, labspath) {
return filepath.Join(lpath, strings.TrimPrefix(path, labspath))
}
return path
}
func lintersFromFlags() map[string]*Linter {
out := map[string]*Linter{}
for _, linter := range config.Enable {
out[linter] = LinterFromName(linter)
}
for _, linter := range config.Disable {
delete(out, linter)
}
if config.Fast {
for _, linter := range slowLinters {
delete(out, linter)
}
}
return out
}
// nolint: gocyclo
func processOutput(state *linterState, out []byte) {
re := state.regex
all := re.FindAllSubmatchIndex(out, -1)
debug("%s hits %d: %s", state.Name, len(all), state.Pattern)
for _, indices := range all {
group := [][]byte{}
for i := 0; i < len(indices); i += 2 {
var fragment []byte
if indices[i] != -1 {
fragment = out[indices[i]:indices[i+1]]
}
group = append(group, fragment)
}
issue := &Issue{Line: 1}
issue.Linter = LinterFromName(state.Name)
for i, name := range re.SubexpNames() {
if group[i] == nil {
continue
}
part := string(group[i])
if name != "" {
state.vars[name] = part
}
switch name {
case "path":
issue.Path = state.fixPath(part)
case "line":
n, err := strconv.ParseInt(part, 10, 32)
kingpin.FatalIfError(err, "line matched invalid integer")
issue.Line = int(n)
case "col":
n, err := strconv.ParseInt(part, 10, 32)
kingpin.FatalIfError(err, "col matched invalid integer")
issue.Col = int(n)
case "message":
issue.Message = part
case "":
}
}
if m, ok := config.MessageOverride[state.Name]; ok {
issue.Message = state.vars.Replace(m)
}
if sev, ok := config.Severity[state.Name]; ok {
issue.Severity = Severity(sev)
} else {
issue.Severity = "warning"
}
if state.exclude != nil && state.exclude.MatchString(issue.String()) {
continue
}
if state.include != nil && !state.include.MatchString(issue.String()) {
continue
}
state.issues <- issue
}
return
}
func findVendoredLinters() string {
gopaths := strings.Split(getGoPath(), string(os.PathListSeparator))
for _, home := range vendoredSearchPaths {
for _, p := range gopaths {
joined := append([]string{p, "src"}, home...)
vendorRoot := filepath.Join(joined...)
if _, err := os.Stat(vendorRoot); err == nil {
return vendorRoot
}
}
}
return ""
}
// Go 1.8 compatible GOPATH.
func getGoPath() string {
path := os.Getenv("GOPATH")
if path == "" {
user, err := user.Current()
kingpin.FatalIfError(err, "")
path = filepath.Join(user.HomeDir, "go")
}
return path
}
// addPath appends p to paths and returns it if:
// 1. p is not a blank string
// 2. p doesn't already exist in paths
// Otherwise paths is returned unchanged.
func addPath(p string, paths []string) []string {
if p == "" {
return paths
}
for _, path := range paths {
if p == path {
return paths
}
}
return append(paths, p)
}
// Ensure all "bin" directories from GOPATH exists in PATH, as well as GOBIN if set.
func configureEnvironment() {
gopaths := strings.Split(getGoPath(), string(os.PathListSeparator))
paths := strings.Split(os.Getenv("PATH"), string(os.PathListSeparator))
gobin := os.Getenv("GOBIN")
if config.VendoredLinters && config.Install {
vendorRoot := findVendoredLinters()
if vendorRoot == "" {
kingpin.Fatalf("could not find vendored linters in GOPATH=%q", getGoPath())
}
debug("found vendored linters at %s, updating environment", vendorRoot)
if gobin == "" {
gobin = filepath.Join(gopaths[0], "bin")
}
// "go install" panics when one GOPATH element is beneath another, so we just set
// our vendor root instead.
gopaths = []string{vendorRoot}
}
for _, p := range gopaths {
paths = addPath(filepath.Join(p, "bin"), paths)
}
paths = addPath(gobin, paths)
path := strings.Join(paths, string(os.PathListSeparator))
gopath := strings.Join(gopaths, string(os.PathListSeparator))
if err := os.Setenv("PATH", path); err != nil {
warning("setenv PATH: %s", err)
}
debug("PATH=%s", os.Getenv("PATH"))
if err := os.Setenv("GOPATH", gopath); err != nil {
warning("setenv GOPATH: %s", err)
}
debug("GOPATH=%s", os.Getenv("GOPATH"))
if err := os.Setenv("GOBIN", gobin); err != nil {
warning("setenv GOBIN: %s", err)
}
debug("GOBIN=%s", os.Getenv("GOBIN"))
}