BN_cmp regards negative and positive zero as different numbers

Bug #1734447 reported by Guido
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
openssl (Ubuntu)
New
Undecided
Unassigned

Bug Description

"BN_cmp() returns -1 if a < b, 0 if a == b and 1 if a > b." ( https://wiki.openssl.org/index.php/Manual:BN_cmp(3) )

The libcrypto.a shipped with Ubuntu makes a distinction between a negative 0 ("-0") and a positive zero ("0"). This means that BN_cmp(-0, 0) returns -1 and BN_cmp(0, -0) returns 1. The latest versions of OpenSSL (1.0.2m and 1.1.0g) both return 0 for both comparisons, which is what you would expect.

lsb_release -rd:

Description: Ubuntu 16.04.3 LTS
Release: 16.04

Tested on both the 32 bit and 64 bit versions of this Ubuntu release.

Attached is a proof of concept.

The anomaly may also be caused by BN_dec2bn rather than BN_cmp.

The bug is unlikely to have direct security consequences, but may cause unexpected behaviour in other applications that depend on this widespread library.

Found with https://github.com/guidovranken/bignum-fuzzer

Revision history for this message
Guido (guidovranken) wrote :
Revision history for this message
Guido (guidovranken) wrote :

This also affects BN_mod_add(-0, -0, 10) (result should be 0, but is 10)

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers