Page 1 of 1
About Decimals... [EDIT]
Posted: Mon Aug 13, 2018 11:41 pm
by Roberto Lopez
Hi All,
Usually I work with two decimals in math operations with no problems.
Now, I need to work with three decimals, converting number to strings and strings to numbers...
The result of the following should be 1.000, but, it is shown as 0.996.
I've played with SET DECIMALS and SET FIXED without results...
What I'm doing wrong?
Code: Select all
N1 := 1/12
CN1 := STR( N1 ,7,3)
VN1 := VAL( CN1 )
N2 := VN1 * 12
MSGINFO( STR( N2 ,7,3) )
If you do this:
Code: Select all
N1 := 1 / 12
N2 := N1 * 12
MSGINFO( STR( N2 ,7,3) )
The result is 1.000
TIA.
Re: About Decimals...
Posted: Tue Aug 14, 2018 12:03 am
by IMATECH
Code: Select all
N1 := Round( 1/12, 3 )
CN1 := Str( N1 ,7,3)
VN1 := Val( CN1 )
N2 := Round( VN1 * 12, 3 )
MSGINFO( STR( N2 ,7,3) )
*Not tested
Re: About Decimals...
Posted: Tue Aug 14, 2018 12:52 am
by Roberto Lopez
IMATECH wrote: ↑Tue Aug 14, 2018 12:03 am
Code: Select all
N1 := Round( 1/12, 3 )
CN1 := Str( N1 ,7,3)
VN1 := Val( CN1 )
N2 := Round( VN1 * 12, 3 )
MSGINFO( STR( N2 ,7,3) )
*Not tested
The result is the same: 0.996
Thanks anyway!
Re: About Decimals...
Posted: Tue Aug 14, 2018 2:12 am
by andyglezl
Maybe this way
FUNCTION ConDeci()
SET FIXED ON
SET DECI TO 15
N1 := 1/12
msgdebug( N1 )
CN1 := ALLTRIM( STR( N1, 15, 12 ) )
msgdebug( CN1 )
VN1 := VAL( CN1 )
msgdebug( VN1 )
SET DECI TO 2
N2 := VN1 * 12
msgdebug( N2 )
msgdebug( STR( N2 ,7,3) )
RETURN Nil
Re: About Decimals...
Posted: Tue Aug 14, 2018 6:40 am
by Roberto Lopez
andyglezl wrote: ↑Tue Aug 14, 2018 2:12 am
Maybe this way
FUNCTION ConDeci()
SET FIXED ON
SET DECI TO 15
N1 := 1/12
msgdebug( N1 )
CN1 := ALLTRIM( STR( N1, 15, 12 ) )
msgdebug( CN1 )
VN1 := VAL( CN1 )
msgdebug( VN1 )
SET DECI TO 2
N2 := VN1 * 12
msgdebug( N2 )
msgdebug( STR( N2 ,7,3) )
RETURN Nil
Thanks Andrés, it works.
Doug answered my question in Harbour users group too.
I'm introducing the problem, limiting the decimals to 3.
I'm doing the conversions, since I'm storing the values in a grid (as string) and then recalling them to store in a table (as numbers).
Thanks again!
Re: About Decimals... [EDIT]
Posted: Tue Aug 14, 2018 9:24 am
by edk
The variable N1: = 1/12 gives the result: 0.08333(3)
When converting:
CN1: = STR (N1, 7, 3)
VN1: = VAL (CN1)
you rounded up to three decimal, which resulted in 0.083
In such a rounded number, multiplying by 12, we get the result: 0.996
Roberto Lopez wrote: ↑Mon Aug 13, 2018 11:41 pm
Code: Select all
N1 := 1/12 //=> 0.083333333
CN1 := STR( N1 ,7,3) //=> " 0.083"
VN1 := VAL( CN1 ) //=> 0.083
N2 := VN1 * 12 //=> 0.996
MSGINFO( STR( N2 ,7,3) ) //=> " 0.996"
See that:
Code: Select all
SET DECIMALS TO 3
N1 := 1/12
VN1 := VAL (STR( N1 ,7,3))
MSGDebug(N1, VN1, N1 == VN1)
Although SET DECIMALS TO 3 is set, the value of variable N1 is stored with the greatest precision. SET DECIMALS has no effect on the result of the calculation, only on its display.
Therefore, the value of N1== VN1 is returned as .F.
SET DECIMALS
Toggle the
console display
Syntax:
SET DECIMALS TO [<nDecimal>]
Arguments:
<nDecimal> Number of decimals places
Description:
This command establishes the number of decimal places that Harbour
will display in mathematical calculations, functions, memory variables, and fields.
Re: About Decimals...
Posted: Tue Aug 14, 2018 6:35 pm
by andyglezl
Roberto Lopez wrote: ↑Tue Aug 14, 2018 6:40 am
Thanks Andrés, it works.
Doug answered my question in Harbour users group too.
I'm introducing the problem, limiting the decimals to 3.
I'm doing the conversions, since I'm storing the values in a grid (as string) and then recalling them to store in a table (as numbers).
Thanks again!
Al contrario Roberto, gracias a ti.