profiling http handler in go lang

Question

I am trying to profile my http handler written in go. Which on every http request download an image from S3, resize it/crop it and write it in response.

I have followed this link and tried to profile my code as mentioned using easy method as well as hard method. Now, when i use the following line as mentioned in the code.

defer profile.Start(profile.CPUProfile).Stop() 

It doesn't write anything in the /tmp/profie[some number]/cpu.pprof file

func main() {                                    

       defer profile.Start(profile.CPUProfile).Stop()   

      if err := http.ListenAndServe(":8081", http.HandlerFunc(serveHTTP)); err != nil {
       logFatal("Error when starting or running http server: %v", err)
    }       
}

func serveHTTP(w http.ResponseWriter, r *http.Request) {
        keyName := r.URL.Path[1:]
        s3Client := s3.New(session.New(), &aws.Config{Region: aws.String(region)})
        params := &s3.GetObjectInput{
        Bucket: aws.String(bucketName),
        Key: aws.String(keyName),
        }

    mw := imagick.NewMagickWand()
    defer mw.Destroy()
    ...
}

Moreover, when i used the defer profile.Start(profile.CPUProfile).Stop() line inside the serveHTTP like :

func serveHTTP(w http.ResponseWriter, r *http.Request) {

    defer profile.Start(profile.CPUProfile).Stop()
    ......
}

It creates multiple files in the /tmp/profile[some number] folder. So, first question is why it is not writing in the file and secondly shouldn't it be places inside the serveHTTP method because server will get started only once. Hence main() will be called once wheres serveHTTP wil be called on every request.

Part 1

 .               124:   s3Client := s3.New(session.New(), &aws.Config{Region: aws.String(region)})
 .          .    125:        params := &s3.GetObjectInput{
 .          .    126:       Bucket: aws.String(masterBucketName),
 .          .    127:       Key: aws.String(keyName),
 .    32.01kB    128:       }
 .          .    129:
 .          .    130:   mw := imagick.NewMagickWand()
 .          .    131:   defer mw.Destroy()
 .          .    132:   
 .          .    133:   out, err := s3Client.GetObject(params)          
 .          .    134:
 .          .    135:   if strings.EqualFold(keyName[strings.LastIndex(keyName,".")+1:len(keyName)], "gif") {
 .          .    136:
 .    40.11kB    137:       blobGiff, err := ioutil.ReadAll(out.Body)
 .          .    138:       w.Header().Set("Content-Type", "image/gif")
 .          .    139:       w.Header().Set("Cache-Control", "max-age: 604800, public")  
 .          .    140:       w.Header().Set("Last-Modified", time.Now().Format(http.TimeFormat))
 .          .    141:       w.Header().Set("Expires", time.Now().AddDate(1, 0, 0).Format(http.TimeFormat))  
 .          .    142:

Part 2 :

                                   else {
         .          .    167:       img, err := ioutil.ReadAll(out.Body)
         .          .    168:       if err != nil {
         .          .    169:          
         .          .    170:          w.WriteHeader(http.StatusNotFound) 
         .     1.56MB    171:          return      
         .          .    172:       }   

Also, in the above two parts line 128, 137 and 171 has memory leaks, right? Also, I don't find any option to close/destroy the s3Client and blobGiff (byte []).

Answer 1

To profile a http server while it's running you can use the net/http/pprof package.

Just add

import _ "net/http/pprof"

to your imports and open http://localhost:8081/debug/pprof/ in your browser.

Answer 2

First of all, use import "net/http/pprof", not import _ "net/http/pprof". The latter one didn't recognize the pprof in the below routes.

I was using the default serveMux/multiplexer. But then I created my own, and, as people suggested, it has performance implications.

myMux := http.NewServeMux()

Then added the route for the request

myMux.HandleFunc("/", serveHTTP)

Morever, I also added the routes for to make the http://localhost:8081/debug/pprof/ work

myMux.HandleFunc("/debug/pprof/", pprof.Index)
myMux.HandleFunc("/debug/pprof/{action}", pprof.Index)
myMux.HandleFunc("/debug/pprof/symbol", pprof.Symbol)

So, the final code would be:

import "net/http/pprof"

func main() {
    myMux := http.NewServeMux()
    myMux.HandleFunc("/", serveHTTP)

    myMux.HandleFunc("/debug/pprof/", pprof.Index)
        myMux.HandleFunc("/debug/pprof/{action}", pprof.Index)
        myMux.HandleFunc("/debug/pprof/symbol", pprof.Symbol)

    if err := http.ListenAndServe(":8081", myMux); err != nil {
        logFatal("Error when starting or running http server: %v", err)
    }
}