Possible memory leak related to crypto pkg usage

Bug #1276224 reported by Herbert
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
goamz
Undecided
Unassigned

Bug Description

I'm debugging a program built with Go 1.2, that does lots of operations in S3 using goamz and MySQL using go-sql-driver.

However, lately it started using too much memory and just growing until it is caught by the OOM killer. I'm not sure when it started, because I made some changes to this program to fix some bugs and after working properly I was cough into this memory issue.

I enabled the pprof http server and got some stats during a test run.

(pprof) top20
Total: 2235.0 MB
  1010.0 45.2% 45.2% 1010.0 45.2% crypto/tls.(*block).reserve
   243.5 10.9% 56.1% 463.5 20.7% crypto/x509.parseCertificate
   189.5 8.5% 64.6% 189.5 8.5% bytes.makeSlice
   170.5 7.6% 72.2% 390.5 17.5% crypto/tls.(*Conn).readHandshake
   147.0 6.6% 78.8% 147.0 6.6% reflect.unsafe_NewArray
   130.0 5.8% 84.6% 130.0 5.8% bufio.NewReaderSize
    80.5 3.6% 88.2% 80.5 3.6% crypto/rc4.NewCipher
    72.5 3.2% 91.4% 72.5 3.2% crypto/x509/pkix.(*Name).FillFromRDNSequence
    46.0 2.1% 93.5% 46.0 2.1% math/big.nat.make
    27.5 1.2% 94.7% 1283.0 57.4% net/http.(*Transport).dialConn
    25.0 1.1% 95.8% 25.0 1.1% encoding/asn1.parsePrintableString
    18.5 0.8% 96.7% 18.5 0.8% runtime.convT2E
    17.0 0.8% 97.4% 17.0 0.8% crypto/sha1.New
    13.0 0.6% 98.0% 30.0 1.3% crypto/hmac.New
     4.0 0.2% 98.2% 4.0 0.2% crypto/x509.(*Certificate).buildChains
     4.0 0.2% 98.4% 50.0 2.2% encoding/asn1.parseBigInt
     3.5 0.2% 98.5% 8.0 0.4% net.socket
     3.5 0.2% 98.7% 485.0 21.7% net/http.(*persistConn).readLoop
     3.5 0.2% 98.8% 3.5 0.2% newdefer
     2.5 0.1% 98.9% 2.5 0.1% concatstring

I disabled HTTPS in the Region connection I'm using and the looks like the problem went away. I'm not sure if this is a Go issue or a goamz one.

Total: 14.0 MB
    10.0 71.4% 71.4% 10.0 71.4% bufio.NewReaderSize
     1.5 10.7% 82.1% 1.5 10.7% net/http.(*persistConn).readLoop
     1.0 7.1% 89.3% 1.0 7.1% fmt.Sprintf
     0.5 3.6% 92.9% 0.5 3.6% net.sockaddrToTCP
     0.5 3.6% 96.4% 0.5 3.6% newdefer
     0.5 3.6% 100.0% 0.5 3.6% runtime.allocm
     0.0 0.0% 100.0% 0.5 3.6% bitbucket.org/companyname/reponame/logger.(*Logger).Printf
     0.0 0.0% 100.0% 10.0 71.4% bufio.NewReader

To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers