|
Post by qbasicdude on Jan 26, 2023 3:19:33 GMT
This happens when scraping X and Y values out of G-Code files as strings then converting them to numbers using val()
The number comes out correctly as a string, but when converted with val() it does not. The problem is intermittent.
to strip it down to the problem area, it's something like:
offset = 0.214
x$ = "9.983" xcoord = val(x$) xcoord=xcoord+offset print #1, "X"; xcoord
The result to the file would occasionally have a very long decimal result like XX.XXX00000000000001. That is not a part of the original string.
I don't know why this is happening, or even how to round it. Normally I round to third decimal like xcoord = cint(xcoord * 1000) / 1000 but the decimal is so long I think it's maxing out the capabilities and giving an error when I try to do that.
There is no problem with the original file that I read the values from.
There is no problem with the offset amount. There is no problem with the number as a string when it is scraped from the g-code line. I tried printing to the output file as a string and it works fine but I can't add the offset. I also tried printing the offset to the file with each line and there is never a problem with it.
|
|
|
Post by bplus on Jan 26, 2023 4:03:16 GMT
Ah the bane of QB64, getting numbers to behave as you want and expect.
Really don't have any control over Val() as coder, maybe a QB64 developer might come up with something more controlable.
Your example happens to work fine without any medicine. But you asked about rounding and here's one approach
Here is the display of a bunch of numbers rounded from 0 to 5 decimal places (dp), doubt it's perfect but gets you closer specially with those sci-notation ones popping in and ruining everything!
_Title "Random Numbers Rounded 0 to 5 Decimal Places" ' b+ 2023-01-25 Screen _NewImage(1200, 700, 32) _PrintMode _KeepBackground Color &HFFBBBBBB For y = 0 To 40 Line (0, y * 16)-(_Width, y * 16), &HFF777777 Next For i = 1 To 40 r = Rnd * 10 ^ (Rnd * 8 - 4) Print r; Tab(20); For digits = 0 To 5 Print _Trim$(Str$(digits)); ": "; Right$(Space$(10) + roundDP$(r, digits), 10); Space$(7); Next Print Next
Function roundDP$ (num, digits) 'mod 2022-11-10 Dim s$, dot s$ = N2S$(Str$(num + (Sgn(num) * .5) * 10 ^ -digits)) dot = InStr(s$, ".") If dot Then roundDP$ = Mid$(s$, 1, dot + digits) Else roundDP$ = s$ End Function
Function N2S$ (EXP$) 'remove scientific Notation to String (~40 LOC) 'SMcNeill Jan 7, 2020 ref: https://www.qb64.org/forum/index.php?topic=1555.msg112989#msg112989 'Last Function in code marked Best Answer (removed debug comments and blank lines added these 2 lines.) ReDim t$, sign$, l$, r$, r&& ReDim dp As Long, dm As Long, ep As Long, em As Long, check1 As Long, l As Long, i As Long t$ = LTrim$(RTrim$(EXP$)) If Left$(t$, 1) = "-" Or Left$(t$, 1) = "N" Then sign$ = "-": t$ = Mid$(t$, 2) dp = InStr(t$, "D+"): dm = InStr(t$, "D-") ep = InStr(t$, "E+"): em = InStr(t$, "E-") check1 = Sgn(dp) + Sgn(dm) + Sgn(ep) + Sgn(em) If check1 < 1 Or check1 > 1 Then N2S = _Trim$(EXP$): Exit Function 'If no scientic notation is found, or if we find more than 1 type, it's not SN! Select Case l 'l now tells us where the SN starts at. Case Is < dp: l = dp Case Is < dm: l = dm Case Is < ep: l = ep Case Is < em: l = em End Select l$ = Left$(t$, l - 1) 'The left of the SN r$ = Mid$(t$, l + 1): r&& = Val(r$) 'The right of the SN, turned into a workable long If InStr(l$, ".") Then 'Location of the decimal, if any If r&& > 0 Then r&& = r&& - Len(l$) + 2 Else r&& = r&& + 1 End If l$ = Left$(l$, 1) + Mid$(l$, 3) End If Select Case r&& Case 0 'what the heck? We solved it already? 'l$ = l$ Case Is < 0 For i = 1 To -r&& l$ = "0" + l$ Next l$ = "." + l$ Case Else For i = 1 To r&& l$ = l$ + "0" Next l$ = l$ End Select N2S$ = sign$ + l$ End Function
For more accuracy try r and num as double type.
This will round to right side of decimal with negative dp ie -1 rounds to nearest 10, -2 to nearest 100, ...
Oh I see I was in my REDIM period, works as well as DIM if not better.
|
|
|
Post by bplus on Jan 26, 2023 18:19:52 GMT
Just had another thought about Val(). It might be improved with the variable Type that Val() function is outputting to?
sn$ = "3.141592654" Dim a As Integer Dim b As Single Dim c As _Float
a = Val(sn$) b = Val(sn$) c = Val(sn$) Print a, b, c Notice rounding is being used in single Type.
|
|
|
Post by bplus on Feb 20, 2023 20:18:46 GMT
|
|