Skip to content

Commit

Permalink
DEVELOPING.md: add pprof instructions
Browse files Browse the repository at this point in the history
Signed-off-by: Silvio Moioli <[email protected]>
  • Loading branch information
moio committed Nov 21, 2022
1 parent 074a466 commit a808f0e
Showing 1 changed file with 35 additions and 0 deletions.
35 changes: 35 additions & 0 deletions DEVELOPING.md
Original file line number Diff line number Diff line change
Expand Up @@ -256,6 +256,41 @@ That being said, downstream `fleet-agent` deployments can perform Kubernetes API
While network traffic is major point of consideration, we also have to consider whether our performance issues are **compute-based**, **memory-based**, or **network-based**.
For example: you may encounter a pod with high compute usage, but that could be caused by heightened network traffic received from the _truly_ malfunctioning pod.

### Using pprof

[http pprof](https://pkg.go.dev/net/http/pprof) handlers are enabled by default with all [default profiles](https://pkg.go.dev/runtime/pprof#Profile) under the `/debug/pprof` prefix.

Additionally, it is possible to enable continuous CPU profiling for `fleetcontroller` to observe how CPU usage changes over time.

Add the following extra Helm values:
```yaml
cpuPprof:
period: "60s"
volumeConfiguration:
hostPath:
path: /tmp/pprof
type: DirectoryOrCreate
```
Notes:
- `period` is the pprof CPU period and can be changed with any other value
- `volumeConfiguration` can be any valid volume configuration (not necessarily `hostPath`)

Alternatively, use the following Helm commandline arguments:
```
--set cpuPprof.period=60s \
--set cpuPprof.volumeConfiguration.hostPath.path=/tmp/pprof \
--set cpuPprof.volumeConfiguration.hostPath.type=DirectoryOrCreate \
```

If using `hostPath`, make sure that the target directory (`/tmp/pprof` in above examples) is writable.

Profiles can be inspected with the [pprof tool](https://github.com/google/pprof), eg.:

```sh
pprof -http=localhost:5000 ./2022-11-04_19_47_18.pprof.fleetcontroller.samples.cpu.pb.gz
```

### Tips and Tricks

Since the Fleet components' controller framework of choice is [Wrangler](https://github.com/rancher/wrangler), we can share caches and avoid unnecessary API requests.
Expand Down

0 comments on commit a808f0e

Please sign in to comment.