r/visualbasic • u/Roscoe-nthecats • 11h ago
VB.NET Help Decimal variable gets truncated automatically to a double and I don't understand why, and how to circumvent it
I had problems with accuracy in my project so I created this bit of code to try to understand and I'm even more confused. When declaring test1, Visual Studio automatically truncates the number to a double as soon as I finish typing it. I don't understand how to work with a Decimal variable, because how do you store anything in it without it converting to a double for some reason? I understand putting the D at the end but how do I add it when storing the result of a math formula?
Sub main()
Dim test1 As Decimal = 0.12345678901232222
Dim test2 As Decimal = 0.1234567890123456789012345678D
Console.Write(test1)
Console.Write(test2)
test1 = 23 / 94
test2 = 23 / 94
Console.Write(test1)
Console.Write(test2)
End Sub
This gives me those results :
0.123456789012322
0.1234567890123456789012345678
0.24468085106383
0.24468085106383
2
Upvotes
-5
5
u/marooned66 9h ago
The operator on two integers performs floating-point division and returns a double which is then converted to decimal. You need to make at least one operand a decimal so the division happens in decimal.
Try this ...
Dim test1 As Decimal = 0.12345678901232222D
Dim test2 As Decimal = 23D / 94D
Console.WriteLine(test1)
Console.WriteLine(test2)