imprecision in snapping to a grid when document units are mm, rev 13918

[Alvin Penner]

    in revs 13868, 13876, 13881, 13896, 13900, 13918, some changes have been made to improve the precision of the viewbox scaling calculation, and the process of snapping to a grid or guide. These changes have worked well for the case of snapping to a grid if the document units are px, see following comment.
    However, if the document units are mm, which is the default, then snapping to a grid is still imprecise. Attached is a demo file. This was obtained from the file default_mm.svg, after adding a grid of 1 mm. Then a multi-line Bezier path was drawn, consisting mostly of lines, more or less randomly, with snapping enabled.
    A significant number of the lines contain round-off error in the last digit. It is worth noting that the error can be either plus or minus, and that it can affect either x or y, so it is essentially random, not related to systematic error in a grid spacing, for example, nor related to turning the page upside down.
    This test was done with Preferences->InputOutput->Numeric Precision set to 8, the default. However a similar result is obtained also at lower settings like 6.

[Alvin Penner (apenner)]

    the problem does not occur if the document units are px. The attached file was produced by loading the file default_px.svg. Then add a grid with a spacing of 5 px. Then set the page height to an integer number of px, in this case 1123. This is to avoid non-integer numbers when inverting the y axis. Then delete any translate transforms that may have acumulated in Layer 1, again to avoid non-integer offsets. Then draw a random set of Bezier lines. The result is attached, using the default numeric precision of 8.
   this drawing contains only integer coordinates, as expected.

[su_v (suv-lp)]

Confirmed with 0.91+devel r13918 on OS X 10.7.5 (this also affects 0.91 when editing a file with viewBox defined in other units than the default user units (CSS Pixels) - e.g. based on the template 'A4 Page' or 'default mm').

[Alvin Penner (apenner)]

    There is an analytical explanation for the round-off error. If we monitor the usage of the routine sp_svg_write_path, in svg-path.cpp, which is responsible for writing data to XML, then we see that the routine is called four times during the creation of a single Bezier path. The first two calls produce data in units of px and the second two produce data in units of mm. Each of these calls involves truncation of the data to the precision specified in the preference value for SVG Output Numeric Precision, normally 8. The problem with this truncation is that it uses the total number of digits both before and after the decimal point, and the number of digits before the decimal point may be different (less) after the unit conversion from px to mm has occurred. Therefore the level of precision required in the converted number (in mm) may be higher than the precision required in the original data (in px). This leads to two classes of data, those which have the same number of leading digits before and after conversion, where we do not anticipate any round-off error problems, and those which have fewer digits before the decimal point after unit conversion, where we have a significant statistical likelihood of encountering round-off error.
    This is illustrated in the attached diagram. The green areas are areas where the unit conversion does not change the number of leading digits. The red areas are areas where a change occurs in both x and y. Snapping a line randomly between and within the green areas does not lead to round-off error, while snapping a line randomly within the red area leads to a high probability of error. In the red area the probability of success, namely that a particular x value or y value will not have error, can be predicted to be s/10, where s is the conversion factor from mm to px, namely 96/25.4. This leads to a probability of success = 38% in the red areas.
    The way to avoid this problem would be to prevent the writing, and truncation, of the original px data to XML, however I don't know how to do that.

[LucaDC (lucadc)]

Alvin Penner wrote:
>The way to avoid this problem would be to prevent the writing, and truncation, of the original px data to XML

What is "original px data"?
If the document unit is mm, 1 px (user unit) = 1 mm.
IMHO the way to avoid this problem is not converting anything at all! All the code that still do this silly thing of thinking in 1/96 of an inch should simply be corrected.