-
Notifications
You must be signed in to change notification settings - Fork 65
src debug command #731
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
src debug command #731
Changes from all commits
Commits
Show all changes
114 commits
Select commit
Hold shift + click to select a range
dfa7592
setup debug command scaffold
DaedalusG e729fa0
added test kubectl events logging
DaedalusG eb9f057
debug: Create ZIP archive
tsenart b48f332
debug: Fail if zip file already exists
tsenart dabaacb
debug: introduce directory structure in zip
tsenart 74a28c3
added --all-naamespaces to get events command
DaedalusG 88abc6f
Merge branch 'main' into src-debugger
DaedalusG 77f06d2
populated with TODO tasks, add get pods output
DaedalusG 746fce8
setting up k8s get pods to enable grabbing logs
DaedalusG 0dff03b
got pod names parsed to slice
DaedalusG b7c4625
pulling logs from all pods, TODO: handle subcontainers
DaedalusG d4821a0
add missing .txt string to log archive names
DaedalusG 3de8d87
used json get pods out, refactored k8s logs into their own spot
DaedalusG 53932e4
refactor get logs into savek8sLogs
DaedalusG fd4ad22
working on logs for previous pods, need to only write when command re…
DaedalusG 9415d56
archives prev-logs if exists
DaedalusG c216923
refactoring getPods to handler scope
DaedalusG e392fbb
refactored all kubectl call functions to be declared outside of handler
DaedalusG 9ef8036
added pod manifests and describes
DaedalusG ae4c240
added some more TODOs
DaedalusG 9d09cb2
added a little error handling on archives
DaedalusG 247d0a3
corrected some error handling
DaedalusG 8115b72
added validation on out flag, and setup for deployment flag
DaedalusG 3631b8a
improved logic for deployment flag
DaedalusG 5db3a39
refactored deployment script
DaedalusG dd52bf3
added get PV and PVC, used out flag as baseDir for archive filestructure
DaedalusG 5c6987c
changed archive filestructure to be oriented around pod directories c…
DaedalusG dc32e6f
cleaned up some finished TODOs, shortened selector values for -d flag
DaedalusG 26ebd78
refactored kubectl archiving into -d flag switch
DaedalusG 53e8ecc
working get containers
DaedalusG 928d61f
Concurrency!!
tsenart 1a655d9
made deploy level kubectl functions concurrent
DaedalusG 572218e
figured out bug in getLogs caused by immediately invoked function and…
DaedalusG 62d915c
all kubectl functions refactored to run as goroutines, bug present ca…
DaedalusG 6943f52
added some prints for debugging
DaedalusG 0336668
clean up some flag validating prints and comments
DaedalusG f26f592
A bunch of improvements
533c545
add todo for logging
DaedalusG 88b2a4e
Merge branch 'src-debugger' of https://github.com/sourcegraph/src-cli…
DaedalusG 2d91a35
fixed hardcoding 999999 in setOpenFileLimits
DaedalusG a3c9977
removed plural k8s func names, started work on docker commands
DaedalusG 709817c
working log archival for docker containers
DaedalusG edbe953
added docker inspect
DaedalusG 419d194
added docker container stats
DaedalusG 8a6aa2a
improved logic for get containers
DaedalusG e2cdff3
cleanup
DaedalusG 877fedc
clarify assumptions
DaedalusG 662c142
introduced debug sub commands kube, comp, and serv -- Ex: src debug k…
DaedalusG b96532a
for some reason my .gitignore was ignoring these new files
DaedalusG 7981e89
changed flagset to use .StringVar rather than .String in flagSet package
DaedalusG b54a1af
correct flag typo in serv
DaedalusG 2d46ea8
fixed verbose output in kube command, changed -out flag to -o
DaedalusG d3d557e
switching to passanger, stating getConfig
DaedalusG e393535
ignore errors and get extsvc configs
7dff3b9
add some comments from work with Tomas
DaedalusG 2e3b525
added namespace flag for kube subcommand
DaedalusG 77f62e0
added a little safety check to the kube command
DaedalusG d1264a9
added a semaphore to kube, added some text safty checks
DaedalusG 47ba3c5
added semphore to all RPC calls in archiveKube
DaedalusG 367de18
added current-context logging before kube command execution
DaedalusG 13a2620
Atempting to debug error inconsistent --previous logs call
DaedalusG a8505f8
added network validation to comp subcommand
DaedalusG 6a64a3c
added a function to pull site config, cleaned up logging and usage funcs
DaedalusG 487ab74
Merge branch 'main' into src-debugger
DaedalusG 9c81467
moved sub-functions to relevant file, trial utility archiveFileFromCo…
DaedalusG d080d66
converted all kube commands to , renamed kube getContainers to getPods
DaedalusG a33444c
refactored compose functions with archiveFileFromCommand
DaedalusG 688ccee
Made archiveFileFromCommand calls aesthetically pleasing, added docke…
DaedalusG e1fe03d
changes all path package calls to path/filepath calls
DaedalusG df05363
emptied graveyard
DaedalusG 9c656bd
return early from failed semaphore.Acquire rather than reading error
DaedalusG 38f5ce1
handle errors, remove direct comparison to booleans
DaedalusG d6d71e5
cleaned up some more linter erros
DaedalusG 023cbe0
refactor adding verify func
DaedalusG 3fafa35
fixed final improperly handled error
DaedalusG 7ecd7de
use semaphore in debug comp
DaedalusG daa38e8
working errgroups in comp
DaedalusG e5c3f8c
refactor to errgroup in kube
DaedalusG 4bdba62
refactored writer in archive functions
DaedalusG 6b35078
corrected all complex filepath calls using fmt.Sprintf()
DaedalusG 94a6538
filter docker ps output to appropriate network
DaedalusG f759711
removed setup debug as will as openfile limit adjustment
DaedalusG 06640e5
normalize site config
8ecdff3
correctly process json
DaedalusG 4396daa
fleshed out serv command
DaedalusG c42c855
Update .gitignore
DaedalusG 9e12b5f
Apply suggestions from code review
DaedalusG ed870e5
implement and repair infered refactors in erics suggestions
DaedalusG dc7aef6
addressed many suggested code improvements for readability and style;…
DaedalusG bd1366c
refactor baseDir processor
DaedalusG 91c28d6
went over errors, still probably needs another run since I widely use…
DaedalusG 109f49e
clarify all string serializations
DaedalusG 877a333
Update cmd/src/debug_comp.go
DaedalusG edf6100
Update cmd/src/debug_common.go
DaedalusG 14f3441
Update cmd/src/debug_common.go
DaedalusG 1b09789
Update cmd/src/debug_kube.go
DaedalusG 7eee487
Update cmd/src/debug_kube.go
DaedalusG 3ab8775
Update cmd/src/debug_kube.go
DaedalusG 066fd39
finishing touches
DaedalusG 10fa745
refactored run command, handled some errors
DaedalusG 0f2f878
bring branch up to main
DaedalusG c54fd23
final error handling cleanup
DaedalusG feb5cf4
add changelog entry
DaedalusG da74d2b
Update cmd/src/debug_serv.go
DaedalusG f0c57a4
Update cmd/src/debug_comp.go
DaedalusG 9a2c7b0
Update cmd/src/debug_kube.go
DaedalusG 4aa5180
Update cmd/src/debug_kube.go
DaedalusG 1a6076b
Update cmd/src/debug.go
DaedalusG 91fef39
address harvey suggestions
DaedalusG 4805a7c
Merge branch 'src-debugger' of https://github.com/sourcegraph/src-cli…
DaedalusG 8eb1d8f
finalize changes from harvey's suggestions
DaedalusG 6b11205
fix go.mod
DaedalusG 48ffc5f
really fix go.mod
DaedalusG cc29b45
Merge branch 'main' into src-debugger
efritz File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,4 +1,6 @@ | ||
./src | ||
./cmd/src/src | ||
*.zip | ||
release | ||
./vendor | ||
.idea | ||
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,42 @@ | ||
package main | ||
|
||
import ( | ||
"flag" | ||
"fmt" | ||
) | ||
|
||
var debugCommands commander | ||
|
||
func init() { | ||
usage := `'src debug' gathers and bundles debug data from a Sourcegraph deployment for troubleshooting. | ||
|
||
Usage: | ||
|
||
src debug command [command options] | ||
|
||
The commands are: | ||
|
||
kube dumps context from k8s deployments | ||
compose dumps context from docker-compose deployments | ||
server dumps context from single-container deployments | ||
|
||
|
||
Use "src debug command -h" for more information about a subcommands. | ||
src debug has access to flags on src -- Ex: src -v kube -o foo.zip | ||
|
||
` | ||
|
||
flagSet := flag.NewFlagSet("debug", flag.ExitOnError) | ||
handler := func(args []string) error { | ||
debugCommands.run(flagSet, "src debug", usage, args) | ||
return nil | ||
} | ||
|
||
// Register the command. | ||
commands = append(commands, &command{ | ||
flagSet: flagSet, | ||
aliases: []string{}, | ||
handler: handler, | ||
usageFunc: func() { fmt.Println(usage) }, | ||
}) | ||
} |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,123 @@ | ||
package main | ||
|
||
import ( | ||
"archive/zip" | ||
"context" | ||
"encoding/json" | ||
"fmt" | ||
"log" | ||
"os" | ||
"path/filepath" | ||
"strings" | ||
|
||
"github.com/sourcegraph/src-cli/internal/exec" | ||
|
||
"github.com/sourcegraph/sourcegraph/lib/errors" | ||
) | ||
|
||
type archiveFile struct { | ||
name string | ||
data []byte | ||
err error | ||
} | ||
|
||
func archiveFileFromCommand(ctx context.Context, path, cmd string, args ...string) *archiveFile { | ||
f := &archiveFile{name: path} | ||
f.data, f.err = exec.CommandContext(ctx, cmd, args...).CombinedOutput() | ||
if f.err != nil { | ||
f.err = errors.Wrapf(f.err, "executing command: %s %s: received error: %s", cmd, strings.Join(args, " "), f.data) | ||
} | ||
return f | ||
} | ||
|
||
// verify prompts the user to confirm they want to run the command | ||
func verify(confirmationText string) (bool, error) { | ||
input := "" | ||
for strings.ToLower(input) != "y" && strings.ToLower(input) != "n" { | ||
fmt.Printf("%s [y/N]: ", confirmationText) | ||
if _, err := fmt.Scanln(&input); err != nil { | ||
return false, err | ||
} | ||
} | ||
|
||
return strings.ToLower(input) == "y", nil | ||
} | ||
|
||
func processBaseDir(base string) (string, string) { | ||
if !strings.HasSuffix(base, ".zip") { | ||
return base + ".zip", base | ||
} | ||
|
||
return base, strings.TrimSuffix(base, ".zip") | ||
} | ||
|
||
// write all the outputs from an archive command passed on the channel to to the zip writer | ||
func writeChannelContentsToZip(zw *zip.Writer, ch <-chan *archiveFile, verbose bool) error { | ||
for f := range ch { | ||
if verbose { | ||
log.Printf("archiving file %q with %d bytes", f.name, len(f.data)) | ||
} | ||
|
||
if f.err != nil { | ||
return f.err | ||
} | ||
|
||
zf, err := zw.Create(f.name) | ||
if err != nil { | ||
return errors.Wrapf(err, "failed to create %q", f.name) | ||
} | ||
|
||
if _, err := zf.Write(f.data); err != nil { | ||
return errors.Wrapf(err, "failed to write to %q", f.name) | ||
} | ||
} | ||
return nil | ||
} | ||
|
||
// TODO: Currently external services and site configs are pulled using the SRC_ENDPOINT env var, | ||
DaedalusG marked this conversation as resolved.
Show resolved
Hide resolved
|
||
// if theres a way to validate that the env var is pointing at the same instance as the docker and kubectl commands, | ||
// it should be implemented. | ||
|
||
// TODO: file issue on the existence of OAuth signKey which needs to be redacted | ||
|
||
// getExternalServicesConfig calls src extsvc list with the format flag -f, | ||
// and then returns an archiveFile to be consumed | ||
func getExternalServicesConfig(ctx context.Context, baseDir string) *archiveFile { | ||
const fmtStr = `{{range .Nodes}}{{.id}} | {{.kind}} | {{.displayName}}{{"\n"}}{{.config}}{{"\n---\n"}}{{end}}` | ||
return archiveFileFromCommand( | ||
ctx, | ||
filepath.Join(baseDir, "config", "external_services.txt"), | ||
os.Args[0], "extsvc", "list", "-f", fmtStr, | ||
) | ||
} | ||
|
||
// getSiteConfig calls src api -query=... to query the api for site config json | ||
func getSiteConfig(ctx context.Context, baseDir string) *archiveFile { | ||
const siteConfigStr = `query { site { configuration { effectiveContents } } }` | ||
f := archiveFileFromCommand(ctx, | ||
filepath.Join(baseDir, "config", "siteConfig.json"), | ||
os.Args[0], "api", "-query", siteConfigStr, | ||
) | ||
|
||
if f.err != nil { | ||
return f | ||
} | ||
|
||
var siteConfig struct { | ||
Data struct { | ||
Site struct { | ||
Configuration struct { | ||
EffectiveContents string | ||
} | ||
} | ||
} | ||
} | ||
|
||
if err := json.Unmarshal(f.data, &siteConfig); err != nil { | ||
f.err = err | ||
return f | ||
} | ||
|
||
f.data = []byte(siteConfig.Data.Site.Configuration.EffectiveContents) | ||
return f | ||
} |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It's unusual in Go to bundle an error field into a structure, as Go tends to strongly prefer returning
error
values as separate return parameters. Normally, the "constructor" would return something like*archiveFile, error
, and let callers handleerr
that way, rather than having to introspect the returned*archiveFile
. This tends to look something like this in practice:Where we do tend to break this in Sourcegraph is where the error actually needs to persist across invocations — the most common case is where a GraphQL resolver does some sort of one time internal call to retrieve things from the database, but then might need to return the same error for each field that was requested in the GraphQL query.
You don't need to change this, but it's worth being aware of for future Go projects.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Will investigate after merge, ty ty
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@LawnGnome FYI these values go through a channel which is why it's packaged as a struct in the first place. Does that change this advice?