Float output with Bitwise operations in PL/SQL

[tekmilind]

Hi,
I am facing some problems with bitwise operators in PL/SQL. The formula i have is : (((x[0]<<8)+x[1])/128) where x[0]=FF. The correct output value is &quot;-.01&quot;. I have translated the formula as (((x[0] * POWER(2, 8))+x[1])/128), but i am not able to get the float output.

Thanks in advance for the help.

- Milind

 

[ThomVF]

What are you getting ?

What types are x[0] and x[1] ?

 

[tekmilind]

I am getting the value as 0. x[0] and x[1] are in hexadecimal. Their values are FF. I am converting that to decimal to be substituted in the formula '(((x[0] * POWER(2, 8))+x[1])/128)'. Since there are no left shift operators in PL/SQL i am converting FF to decimal.

Let me know if this information helps.

Regards,
Milind

 

[ThomVF]

Ok, slap me if I'm an idiot but I'm good at playing the devil's advocate:

If I subsititue 255 (FF in decimal) for x[0] and x[1], using this formula:

select (((255 * POWER(2,8)) + 255) /128)
from dual;

I get 511.99218 not 0. I might be doing this wrong..

You are using PL/SQL. Are you selecting the result of this formula into a variable? What type of variable is it?

 

[tekmilind]

You are absolutely correct. But here is my problem the actual formula is &quot;(((x[0]<<8)+x[1])/128) where x[0]=FF and x[1] is FF. The expected output value is &quot;-.01&quot;. Since oracle does not have left shift and right shift i have translated the formula to '(((x[0] * POWER(2, 8))+x[1])/128)'. I am just trying to run it as a SQL. The reason i got zero was, i used a bitand function to &quot;and&quot; the result with 255.

- Milind

 

[ThomVF]

Not sure of the validitity of this, but I found a guy who wrote an LSHIFT in PL/SQL.

Check it out...

http://www.gt.ed.net/keith/plsql/fast_bw.txt

Called from this page...

http://www.gt.ed.net/keith/plsql/index.html

Tom

 

[tekmilind]

Thanks for the help. I am not still not able to get the output of -0.01. I have decided to use JAVA instead to do the biwise operations. Thanks a lot for all the help.

Regards,
Milind