Getting Started with Adobe After Effects - Part 6: Motion Blur


Upload Image Close it
Select File

Browse by Tags · View All
SQL 3
COLUMNPROPERTY 1
OBJECT_NAME 1
OBJECT_ID 1
SysColumns 1
IF EXISTS 1
decimal 1
float 1
conversion error 1
implicit convertion 1

Archive · View All
March 2012 3
April 2012 2

Adam Tokarski's Blog

Float trap

Apr 10 2012 12:00AM by Adam Tokarski   

We all know that float is type just approximate value. There is how can we use that to set up a trap for ourselves.

Some times ago, I've wrote some script aggregating measures, that was - at one moment - grouping after values.
Just imagine my surprise, when I found that in resultset were two, different, values 457.84!
As I worked it out, in my temp table, created by

SELECT INTO
that problematic VALUE column was assigned FLOAT type.

That both values 457.84 was, indeed, numbers with different digit at 15th place after decimal!

Generally, for strict comparison, FLOAT (REAL too) is poorly suited. To see it clearly, try:

DECLARE @f FLOAT
SET @f = 1.223
SELECT @f, CAST(@f AS DECIMAL(38, 30)), CAST(@f+1 AS DECIMAL(38, 30))

Thank you for your attention.
-- Adam Tokarski --

Tags: float, decimal


Adam Tokarski
58 · 3% · 1021
4
 
0
Lifesaver
 
0
Refreshed
 
0
Learned
 
0
Incorrect



Submit

Your Comment


Sign Up or Login to post a comment.

"Float trap" rated 5 out of 5 by 4 readers
Float trap , 5.0 out of 5 based on 4 ratings
    Copyright © Rivera Informatic Private Ltd Contact us      Privacy Policy      Terms of use      Report Abuse      Advertising      [ZULU1097]