Profiling Go with pprof: Finding CPU, Memory, and Goroutine Bottlenecks
1. Overview
When a Go service runs slowly or consumes excessive RAM, guesswork usually wastes time and often misses the mark. Go comes with the built-in pprof tool in its standard library. This tool tells you exactly which functions consume CPU, where memory is allocated, and which goroutines are stuck.
After reading this article, you’ll know how to enable pprof for HTTP services, collect profiles from benchmarks and CLI programs, read results using go tool pprof, and compare two profiles to verify an optimization.
2. What is pprof and What Types of Profiles Are There?
pprof consists of two parts:
- Collection side:
runtime/pprofandnet/http/pprofpackages in the standard library. They write profiles in protobuf format. - Analysis side:
go tool pprofcommand. It reads profile files or URLs and displays results as tables, source code, or web interface.
Common profile types:
| Profile | Measures | Enabled by default? |
|---|---|---|
profile (CPU) |
CPU time per function, sampled 100 times/second | Yes, when requested |
heap |
Live memory and total allocated | Yes |
allocs |
Same as heap, but defaults to viewing total allocations |
Yes |
goroutine |
Stack trace of all current goroutines | Yes |
block |
Wait time on channels, select, sync.Cond |
No |
mutex |
Contention on sync.Mutex/RWMutex |
No |
trace |
Detailed event trace (use go tool trace) |
Yes, when requested |
3. Enable pprof for HTTP Services
The fastest way is to import net/http/pprof to get its side effect. This package registers /debug/pprof/* handlers to http.DefaultServeMux.
package main
import (
"log"
"net/http"
_ "net/http/pprof" // registers /debug/pprof/* to DefaultServeMux
)
func main() {
// Separate pprof server, only listens on localhost
go func() {
log.Println(http.ListenAndServe("localhost:6060", nil))
}()
// Main application server uses separate mux, doesn't expose pprof
appMux := http.NewServeMux()
appMux.HandleFunc("GET /invoice", invoiceHandler)
log.Fatal(http.ListenAndServe(":8080", appMux))
}
A few things to note:
- The line
_ "net/http/pprof"only registers handlers; you don’t call any functions in this package. - The pprof server passes
nilas the handler, meaning it usesDefaultServeMux, where the pprof routes are registered. - The main application uses
appMuxseparately, so port8080doesn’t have/debug/pprof.
Quick verification:
curl -s http://localhost:6060/debug/pprof/ | head -n 20
Note: Never expose
/debug/pprofto the Internet. This endpoint reveals stack traces and code structure. Attackers could also use it to overload your service (a 30-second CPU profile does consume resources). Always bind tolocalhost, or place behind an internal network and authentication layer.
4. Collecting and Reading CPU Profiles
Suppose the service has an invoice-building function like this:
type Item struct {
SKU string
Price float64
Qty int
}
type Order struct {
ID int
Items []Item
}
func BuildInvoice(o Order) string {
s := ""
for _, it := range o.Items {
s += fmt.Sprintf("%s x%d = %.2f\n", it.SKU, it.Qty, it.Price*float64(it.Qty))
}
return s
}
At each loop iteration, s += ... creates a new string and copies all old content over. With orders having many items, the cost grows as O(n²).
While the service is handling load (for example, you’re using hey or wrk to send requests to /invoice), collect a CPU profile for 30 seconds:
go tool pprof "http://localhost:6060/debug/pprof/profile?seconds=30"
After collection completes, you enter interactive mode. Type top:
(pprof) top 8
Showing nodes accounting for 2.41s, 85.46% of 2.82s total
flat flat% sum% cum cum%
0.92s 32.62% 32.62% 0.92s 32.62% runtime.memmove
0.41s 14.54% 47.16% 1.38s 48.94% runtime.concatstrings
0.33s 11.70% 58.87% 0.33s 11.70% runtime.memclrNoHeapPointers
0.27s 9.57% 68.44% 0.71s 25.18% runtime.mallocgc
0.18s 6.38% 74.82% 0.52s 18.44% fmt.(*pp).doPrintf
0.13s 4.61% 79.43% 0.13s 4.61% strconv.fmtF
0.10s 3.55% 82.98% 2.47s 87.59% main.BuildInvoice
0.07s 2.48% 85.46% 0.21s 7.45% runtime.scanobject
How to read the two most important columns:
- flat: CPU time spent directly in that function.
- cum (cumulative): time of that function plus all functions it calls.
Here main.BuildInvoice has small flat but cum is close to 88%. Most time falls into memmove and concatstrings, meaning string concatenation. To see details line by line, use list:
(pprof) list BuildInvoice
ROUTINE ======================== main.BuildInvoice in /app/invoice.go
100ms 2.47s (flat, cum) 87.59% of Total
. . 18:func BuildInvoice(o Order) string {
. . 19: s := ""
10ms 10ms 20: for _, it := range o.Items {
90ms 2.46s 21: s += fmt.Sprintf("%s x%d = %.2f\n", ...)
. . 22: }
Line 21 is the culprit. If you want to see a flame graph, open the web interface:
go tool pprof -http=:8081 "http://localhost:6060/debug/pprof/profile?seconds=30"
Your browser will open with views for Top, Graph, Flame Graph, and Source. In the flame graph, the wider the block, the more CPU it consumes.
Fix using strings.Builder and fmt.Fprintf:
func BuildInvoiceFast(o Order) string {
var b strings.Builder
b.Grow(len(o.Items) * 32) // pre-allocate to avoid multiple grows
for _, it := range o.Items {
fmt.Fprintf(&b, "%s x%d = %.2f\n", it.SKU, it.Qty, it.Price*float64(it.Qty))
}
return b.String()
}
5. Heap Profile: Who Is Holding Memory?
go tool pprof http://localhost:6060/debug/pprof/heap
A heap profile contains four “sample index” options. You select the index with the -sample_index flag:
inuse_space(default): number of bytes currently alive at the time of the last GC. Use this to find memory leaks.inuse_objects: number of currently alive objects.alloc_space: total bytes allocated since the program started. Use this to find GC pressure sources.alloc_objects: total number of objects allocated.
go tool pprof -sample_index=alloc_space http://localhost:6060/debug/pprof/heap
(pprof) top 3
flat flat% sum% cum cum%
1.84GB 91.20% 91.20% 1.84GB 91.20% main.BuildInvoice
0.12GB 5.95% 97.15% 0.12GB 5.95% fmt.Sprintf
0.03GB 1.49% 98.64% 0.03GB 1.49% encoding/json.Marshal
The result shows BuildInvoice has allocated 1.84GB total, even though the live amount might be very small. This is a sign that GC has to work hard. This situation matches the amount of runtime.mallocgc and runtime.scanobject you see in the CPU profile from section 4.
Note: Heap profile is also sampled data (by default about 1 sample per 512KB allocation), so the numbers are just estimates. Don’t compare byte-by-byte; look at the ratios.
6. Goroutine Profile: Detecting Goroutine Leaks
If the goroutine count keeps increasing, you can check quickly through a browser or curl without needing go tool pprof:
curl -s "http://localhost:6060/debug/pprof/goroutine?debug=1" | head -n 15
goroutine profile: total 10432
10398 @ 0x43a5d6 0x4071c5 0x406d78 0x6b2f45 0x46c0e1
# 0x6b2f44 main.(*PaymentService).notify+0x64 /app/payment.go:57
There are 10,398 goroutines stuck at the same place in payment.go:57. This is almost certainly a leak. The code causing the error usually looks like:
func (s *PaymentService) notify(ctx context.Context, p Payment) {
result := make(chan error) // unbuffered
go func() {
result <- s.gateway.Send(p) // stuck forever if no one receives
}()
select {
case err := <-result:
log.Println(err)
case <-ctx.Done():
return // inner goroutine gets abandoned
}
}
When ctx expires first, no one reads from result, so the inner goroutine gets stuck permanently at the send statement. The simple fix is to use make(chan error, 1) so the send never blocks.
With debug=2, the endpoint prints full stack traces of each goroutine, including wait times (e.g., [chan send, 42 minutes]). This information is very useful when investigating deadlocks.
7. Block and Mutex Profiles
These two profiles are disabled by default because they have overhead. You enable them in main:
import "runtime"
func init() {
// Record on average 1 mutex contention event per 5
runtime.SetMutexProfileFraction(5)
// Sample 1 block event for every 10µs of blocking
runtime.SetBlockProfileRate(10_000)
}
Then collect profiles as usual:
go tool pprof http://localhost:6060/debug/pprof/mutex
go tool pprof http://localhost:6060/debug/pprof/block
Mutex profile tells which locks cause goroutines to queue up waiting for each other, like a sync.Mutex protecting a shared cache map[int]Customer. Block profile tells which channels or select statements cause goroutines to wait a long time.
Note:
SetBlockProfileRate(1)records every block event. This level is only suitable for local debugging. On production, use a larger rate, or enable temporarily.
8. Profiling from Benchmarks
For a specific function, the simplest way is to write a benchmark and let go test generate profiles. The code below uses b.Loop() which requires Go 1.24+.
package main
import (
"fmt"
"testing"
)
func sampleOrder(n int) Order {
items := make([]Item, n)
for i := range items {
items[i] = Item{SKU: fmt.Sprintf("SKU-%04d", i), Price: 19.99, Qty: i%5 + 1}
}
return Order{ID: 1, Items: items}
}
func TestBuildInvoiceFastMatchesOriginal(t *testing.T) {
order := sampleOrder(200)
if got, want := BuildInvoiceFast(order), BuildInvoice(order); got != want {
t.Fatalf("outputs differ:\n got=%q\nwant=%q", got, want)
}
}
func BenchmarkBuildInvoice(b *testing.B) {
order := sampleOrder(500)
b.ReportAllocs()
for b.Loop() {
BuildInvoice(order)
}
}
func BenchmarkBuildInvoiceFast(b *testing.B) {
order := sampleOrder(500)
b.ReportAllocs()
for b.Loop() {
BuildInvoiceFast(order)
}
}
The TestBuildInvoiceFastMatchesOriginal test ensures the optimized version produces identical results to the original. Don’t skip this step: optimizing with incorrect results is pointless. Run the benchmark with profiles:
go test -run='^$' -bench=BuildInvoice -cpuprofile=cpu.out -memprofile=mem.out
BenchmarkBuildInvoice-8 1543 772410 ns/op 6342817 B/op 1002 allocs/op
BenchmarkBuildInvoiceFast-8 19821 60318 ns/op 36472 B/op 504 allocs/op
The strings.Builder version is about 12x faster and allocates about 170x less. The -run='^$' flag skips regular tests to run only benchmarks. The cpu.out and mem.out files are opened with:
go tool pprof -http=:8081 cpu.out
9. Profiling CLI Programs with runtime/pprof
For tools that run once and exit, like an order import job, you use runtime/pprof directly:
package main
import (
"log"
"os"
"runtime"
"runtime/pprof"
)
func main() {
cpuFile, err := os.Create("cpu.out")
if err != nil {
log.Fatal(err)
}
defer cpuFile.Close()
if err := pprof.StartCPUProfile(cpuFile); err != nil {
log.Fatal(err)
}
defer pprof.StopCPUProfile()
importOrders("orders.csv") // main work
memFile, err := os.Create("mem.out")
if err != nil {
log.Fatal(err)
}
defer memFile.Close()
runtime.GC() // update inuse stats before writing
if err := pprof.WriteHeapProfile(memFile); err != nil {
log.Fatal(err)
}
}
Two common mistakes here:
- Calling
log.Fataloros.ExitafterStartCPUProfilewill skip defers. ThenStopCPUProfilewon’t run and the profile file will be empty or corrupted. - Forgetting
runtime.GC()beforeWriteHeapProfile. Theinuse_*stats are only updated after each GC, so the profile might reflect stale state.
10. Comparing Two Profiles
After optimizing, you should measure again rather than trusting your gut. The -diff_base flag subtracts the old profile from the new one:
go tool pprof -diff_base=cpu_before.out cpu_after.out
(pprof) top 3
flat flat% sum% cum cum%
-0.89s 31.56% 31.56% -0.89s 31.56% runtime.memmove
-0.40s 14.18% 45.74% -1.35s 47.87% runtime.concatstrings
-0.25s 8.87% 54.61% -0.66s 23.40% runtime.mallocgc
Negative values mean time has decreased. For the results to be meaningful, both profiles need to be collected under the same conditions: same load, same duration, same machine.
Additionally, since Go 1.21, CPU profiles also serve Profile-Guided Optimization (PGO). You collect a representative CPU profile from production, name it default.pgo in the main package directory, then go build will automatically use that profile to inline and optimize hot code paths. The improvement is typically a few percent, and you don’t have to modify any code.
11. Common Mistakes and When Not to Use
- Profiling too briefly or without load. CPU profile samples at 100Hz. If you only collect for 2 seconds while the service is idle, you’ll only see
runtimeand noise. - Confusing
flatwithcum. A function with highcumisn’t necessarily the culprit; it might just call the culprit. - Confusing
inuse_spacewithalloc_space. To find leaks, look atinuse. To reduce GC pressure, look atalloc. - Measuring I/O with CPU profile. Time goroutines spend waiting for databases or HTTP doesn’t consume CPU so it doesn’t appear in CPU profile. Use block profile,
go tool trace, or distributed tracing. - Leaving pprof public on your application port. See section 3 again.
Note: pprof answers the question “where is time being spent”, not “should we optimize”. If an endpoint already meets its SLA and a function only uses 2% CPU, rewriting it to be more complex usually isn’t worth it.
12. Conclusion
In this article, you:
- safely enabled
net/http/pprofon a separate server atlocalhost; - read CPU profiles using
topandlist, and distinguishedflatfromcum; - used heap profile with
inuse_spaceandalloc_space; - detected goroutine leaks via
debug=1anddebug=2; - enabled block and mutex profiles;
- generated profiles from benchmarks and CLI programs;
- compared before/after with
-diff_base.
The most efficient workflow remains: measure → find hot spots → fix → write tests to preserve behavior → measure again. Since pprof is built into the standard library, you don’t need to install anything to get started.