sb-ext:octets-to-string is sufficiently slower than flexi-streams:octets-to-string.

[William Halliburton]

There may be gold in thar hills...

(defparameter *sample-octets*
  (coerce
   (iter (for i from 1 to 1000000)
         (collect (random 128)))
   '(vector (unsigned-byte 8))))

(define-benchmark sbcl-octets-to-string
  (sb-ext:octets-to-string *sample-octets*))

(define-benchmark fs-octets-to-string
  (flexi-streams:octets-to-string *sample-octets*))

> (benchmark-fs-octets-to-string)
REAL-TIME-MS 2
USER-RUN-TIME-US 2000
SYSTEM-RUN-TIME-US 1000
GC-RUN-TIME-MS 0
PROCESSOR-CYCLES 5966766
BYTES-CONSED 400032

> (benchmark-sbcl-octets-to-string)
REAL-TIME-MS 10
USER-RUN-TIME-US 9998
SYSTEM-RUN-TIME-US 1000
GC-RUN-TIME-MS 0
PROCESSOR-CYCLES 25453260
BYTES-CONSED 2490944

[Nathan Froyd (froydnj)]

If you're converting to UTF-8 (the benchmark really ought to say...), this is partly because SBCL is much more strict about well-formedness checking (at least it was last time I looked) and partly because flexi-streams computes the output length, enabling easy pre-allocation of the output buffer.

[William Halliburton (whalliburton)]

Looks like the speed difference is still there for ascii.

> (benchmark-sbcl-octets-to-string-ascii)
REAL-TIME-MS 116
USER-RUN-TIME-US 110983
SYSTEM-RUN-TIME-US 3999
GC-RUN-TIME-MS 20
PROCESSOR-CYCLES 277290837
BYTES-CONSED 20771984

> (benchmark-fs-octets-to-string-ascii)
REAL-TIME-MS 27
USER-RUN-TIME-US 25996
SYSTEM-RUN-TIME-US 1000
GC-RUN-TIME-MS 0
PROCESSOR-CYCLES 63543294
BYTES-CONSED 4016384

> (benchmark-sbcl-octets-to-string-utf8)
REAL-TIME-MS 108
USER-RUN-TIME-US 98985
SYSTEM-RUN-TIME-US 7999
GC-RUN-TIME-MS 10
PROCESSOR-CYCLES 259652187
BYTES-CONSED 20770832

> (benchmark-fs-octets-to-string-utf8)
REAL-TIME-MS 38
USER-RUN-TIME-US 33994
SYSTEM-RUN-TIME-US 2000
GC-RUN-TIME-MS 0
PROCESSOR-CYCLES 90630252
BYTES-CONSED 4008128

(defparameter *sample-octets*
  (coerce
   (iter (for i from 1 to 1000000)
         (collect (random 128)))
   '(vector (unsigned-byte 8))))

(define-benchmark sbcl-octets-to-string-ascii
  (sb-ext:octets-to-string *sample-octets* :external-format :ascii))

(define-benchmark fs-octets-to-string-ascii
  (flexi-streams:octets-to-string *sample-octets* :external-format :ascii))

(define-benchmark sbcl-octets-to-string-utf8
  (sb-ext:octets-to-string *sample-octets* :external-format :utf8))

(define-benchmark fs-octets-to-string-utf8
  (flexi-streams:octets-to-string *sample-octets* :external-format :utf8))

[Nathan Froyd (froydnj)]

Huh, so it does. Maybe flexi-streams has gotten faster.

Anyway, most of the time is spent in REPLACE, which could be easily sped up the way ADJUST-ARRAY uses it. ADJUST-ARRAY should also be taught more smarts about the "realloc"-esque case.

Or the octets code should be rewritten to be more like flexi-streams.