babl: decrease tolerance

The following conversion is undesirable, going via linear u16 on the
way to a different TRC u16 amount of quantization should be considered
bad enough to be avoided.

chosen RGBA u32 to R'G'B'A u16: steps: 4 error: 0.000005 cost: 876.000000
	/usr/local/lib/babl-0.1/u32.so 0: RGBA u32 to RGBA u16
	/usr/local/lib/babl-0.1/gggl-table-lies.so 0: RGBA u16 to RGBA float
	/usr/local/lib/babl-0.1/sse2-float.so 0: RGBA float to R'G'B'A float
	/usr/local/lib/babl-0.1/gggl.so 0: R'G'B'A float to R'G'B'A u16

This is slower but more correct:

chosen RGBA u32 to R'G'B'A u16: steps: 3 error: 0.000000 cost: 1051.000000
	/usr/local/lib/babl-0.1/simple.so 0: RGBA u32 to RGBA float
	/usr/local/lib/babl-0.1/sse2-float.so 0: RGBA float to R'G'B'A float
	/usr/local/lib/babl-0.1/gggl.so 0: R'G'B'A float to R'G'B'A u16

diff --git a/babl/babl-fish-path.c b/babl/babl-fish-path.c
index c3a22892ee2a8ffc4003e0fa73b2dcf7bfeb9111..2524bb4c753e76e891a6d4b1717352d2c01560d9 100644 (file)
--- a/babl/babl-fish-path.c
+++ b/babl/babl-fish-path.c
@@ -21,7 +21,7 @@
 #include "babl-internal.h"
 #include "babl-ref-pixels.h"
 
-#define BABL_TOLERANCE             0.000005
+#define BABL_TOLERANCE             0.0000047
 #define BABL_MAX_COST_VALUE        2000000
 #define BABL_HARD_MAX_PATH_LENGTH  8
 #define BABL_MAX_NAME_LEN          1024