Possible memory leak related to crypto pkg usage

Bug #1276224 reported by Herbert
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
goamz
New
Undecided
Unassigned

Bug Description

I'm debugging a program built with Go 1.2, that does lots of operations in S3 using goamz and MySQL using go-sql-driver.

However, lately it started using too much memory and just growing until it is caught by the OOM killer. I'm not sure when it started, because I made some changes to this program to fix some bugs and after working properly I was cough into this memory issue.

I enabled the pprof http server and got some stats during a test run.

(pprof) top20
Total: 2235.0 MB
  1010.0 45.2% 45.2% 1010.0 45.2% crypto/tls.(*block).reserve
   243.5 10.9% 56.1% 463.5 20.7% crypto/x509.parseCertificate
   189.5 8.5% 64.6% 189.5 8.5% bytes.makeSlice
   170.5 7.6% 72.2% 390.5 17.5% crypto/tls.(*Conn).readHandshake
   147.0 6.6% 78.8% 147.0 6.6% reflect.unsafe_NewArray
   130.0 5.8% 84.6% 130.0 5.8% bufio.NewReaderSize
    80.5 3.6% 88.2% 80.5 3.6% crypto/rc4.NewCipher
    72.5 3.2% 91.4% 72.5 3.2% crypto/x509/pkix.(*Name).FillFromRDNSequence
    46.0 2.1% 93.5% 46.0 2.1% math/big.nat.make
    27.5 1.2% 94.7% 1283.0 57.4% net/http.(*Transport).dialConn
    25.0 1.1% 95.8% 25.0 1.1% encoding/asn1.parsePrintableString
    18.5 0.8% 96.7% 18.5 0.8% runtime.convT2E
    17.0 0.8% 97.4% 17.0 0.8% crypto/sha1.New
    13.0 0.6% 98.0% 30.0 1.3% crypto/hmac.New
     4.0 0.2% 98.2% 4.0 0.2% crypto/x509.(*Certificate).buildChains
     4.0 0.2% 98.4% 50.0 2.2% encoding/asn1.parseBigInt
     3.5 0.2% 98.5% 8.0 0.4% net.socket
     3.5 0.2% 98.7% 485.0 21.7% net/http.(*persistConn).readLoop
     3.5 0.2% 98.8% 3.5 0.2% newdefer
     2.5 0.1% 98.9% 2.5 0.1% concatstring

I disabled HTTPS in the Region connection I'm using and the looks like the problem went away. I'm not sure if this is a Go issue or a goamz one.

Total: 14.0 MB
    10.0 71.4% 71.4% 10.0 71.4% bufio.NewReaderSize
     1.5 10.7% 82.1% 1.5 10.7% net/http.(*persistConn).readLoop
     1.0 7.1% 89.3% 1.0 7.1% fmt.Sprintf
     0.5 3.6% 92.9% 0.5 3.6% net.sockaddrToTCP
     0.5 3.6% 96.4% 0.5 3.6% newdefer
     0.5 3.6% 100.0% 0.5 3.6% runtime.allocm
     0.0 0.0% 100.0% 0.5 3.6% bitbucket.org/companyname/reponame/logger.(*Logger).Printf
     0.0 0.0% 100.0% 10.0 71.4% bufio.NewReader

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.