Welcome guest. Before posting on our computer help forum, you must register. Click here it's easy and free.

Author Topic: Function Call argument list parsing...  (Read 4546 times)

0 Members and 1 Guest are viewing this topic.

BC_Programmer

    Topic Starter

    Mastermind
  • Typing is no substitute for thinking.
  • Thanked: 1140
    • Yes
    • Yes
    • BC-Programming.com
  • Certifications: List
  • Computer: Specs
  • Experience: Beginner
  • OS: Windows 11
Function Call argument list parsing...
« on: April 20, 2009, 08:23:21 AM »
Over the course of  it's development, my Expression evaluator has consistently had issues with it's "ParseArguments" routine, shown below. The idea is, given a string such as "X,14,Sin(12)*56/i,12", which has already been ripped out of a function call parameter list elsewhere in my code, to parse it and return a array of strings representing each argument. In this case, the return would be an array:

0= "X"
1= "14"
2= "Sin(12)*56/i"
3= "12"


The issue here is that I am constantly, running into subtle bugs in the edge cases (single argument items, for example) and throughout it's existence I have added kludges to get it working each time. So my question is- is there not an easier way to do this? It works- but is there something I'm doing that causes me to require all these kludges?


Also: ARGSEP is a  constant that is set to ","
Code: [Select]
Public Function ParseArguments(ByVal FromString As String, Optional ByVal BracketStart As String = DefaultBracketStart, _
Optional ByVal Bracketend As String = DefaultBracketEnd, Optional ByVal ARGUMENTSEP As String = ARGSEP, Optional ByRef startpos As Long = 1) As String()

    'NOTE: this routine REALLY needs to be rewritten!
    'full of hacks and kludges...

    'ParseArguments Function:
    'assumes the given string is a parameter lists.
    'keeping track of parentheses and quotes, it finds the next ARGSEP that is not part of some other token.
   
   
   
    Dim CurrPos As Long
    Dim SplMake() As String
    Dim CurrArgument As Long, Char As String
    Dim inquote As Boolean, BracketCount As Long
    Dim ArgStart As Long
    'If there isn't even a separator in the string, at all, return the string as the single argument
    If InStr(FromString, ARGUMENTSEP) = 0 Then
        ReDim SplMake(0)
        SplMake(0) = FromString
        ParseArguments = SplMake
        Exit Function
   
    End If
   
   
    If Len(FromString) <= 0 Then
        Erase SplMake()
       
        ParseArguments = SplMake
        Exit Function
    End If
   
   
   
    CurrArgument = 0
    CurrPos = startpos
    ArgStart = CurrPos
    Do
        Char = Mid$(FromString, CurrPos, 1)
        Select Case True
        Case Char = """"
            inquote = Not inquote
   
        Case inquote And CurrPos < Len(FromString)
            'ignore as long as we are in a string.
           
        Case InStr(BracketStart, Char) <> 0 And Char <> ""
            BracketCount = BracketCount + 1
        Case InStr(Bracketend, Char) <> 0 And Char <> ""
       
            BracketCount = BracketCount - 1
               If BracketCount < 0 Then
                   'Sigh.
                   BracketCount = 0
                   
               End If
               
         
        Case InStr(CurrPos, FromString, ARGUMENTSEP) = 0
            ReDim Preserve SplMake(CurrArgument)
            SplMake(CurrArgument) = Mid$(FromString, CurrPos)
            Exit Do
        Case Char = ARGUMENTSEP, (CurrPos >= Len(FromString))
            'Stop
            'If ((Not Inquote) Or _
             (CurrPos >= Len(FromString))) And BracketCount = 0 Then
             If ((Not inquote) And BracketCount = 0) Or (CurrPos >= Len(FromString)) Then
                'wow, that is very strange, heh?
                'This will only be true when inquote=false (0) and bracketcount=0 as well.
                ReDim Preserve SplMake(CurrArgument)
                SplMake(CurrArgument) = Mid$(FromString, ArgStart, Abs(CurrPos - ArgStart))
                If right$(SplMake(CurrArgument), 1) = ARGUMENTSEP Then
                    SplMake(CurrArgument) = Mid$(SplMake(CurrArgument), 1, Len(SplMake(CurrArgument)) - 1)
                End If
                CurrArgument = CurrArgument + 1
                ArgStart = CurrPos + 1
               
                If CurrPos >= Len(FromString) Then
'                    If InStr(1, Bracketend, right$(SplMake(CurrArgument - 1), 1), vbTextCompare) <> 0 And BracketCount <= 0 Then
'                        SplMake(CurrArgument - 1) = Mid$(SplMake(CurrArgument - 1), 1, Len(SplMake(CurrArgument - 1)) - 1)
'                    End If
                   
               
                    Exit Do
                   
                End If
            End If
       
        End Select
   
        CurrPos = CurrPos + 1
    Loop
    If UBound(SplMake) = 0 Then
        If SplMake(0) = "" Then
            'there weren't any arguments- return an actual "Empty" array to denote this to the caller.
            Erase SplMake
        End If
    End If
    startpos = CurrPos
    ParseArguments = SplMake
End Function
I was trying to dereference Null Pointers before it was cool.

Geek-9pm


    Mastermind
  • Geek After Dark
  • Thanked: 1026
    • Gekk9pm bnlog
  • Certifications: List
  • Computer: Specs
  • Experience: Expert
  • OS: Windows 10
Re: Function Call argument list parsing...
« Reply #1 on: April 22, 2009, 03:39:20 PM »
As usual, I have no idea of what you are doing. I do better with 'top down' programming. Perhaps you are using queue/stack solution.
This item: "Sin(12)*56/i"
would suggest that yup need an expression evaluator and maybe a floating pint match kit. I will comment on the expression evaluator. This kind of expression is often called a left to right evaluation, but with FORTRAN rules. Which means that he parser is cgoing to to do some look ahead or even read the thing right to left.
But going going left to right, we parse SIM and determine that is a function we know, will put a token, perhaps a pointer, into a queue (read stack) to identify that item. Next we look inside the left _( and find a constant and next  find the right_) Now can call  SIM with the value. Next find an atom the denotes multiply, so will queue that with the value return by SIM. We find a constant and an atom for divide, lower precedence, so perform what we now have in the queue. Note that divide is coming. Queue it. Next is looks like a variable, go get its value... and so one.

Anyway. I would use an expression evaluator made by somebody else But this is not new stuff. Was that not done bu Egyptians on clay tablets?
Sorry.. I  can not take anymore... I need a Tylenol.


Reno



    Hopeful
  • Thanked: 32
    Re: Function Call argument list parsing...
    « Reply #2 on: April 23, 2009, 04:51:45 AM »
     ::) ::) ::) ::)

    why dont you use split function?

    Code: [Select]
    msg = "SPLIT" & vbCrLf
    s = "X,14,Sin(12)*56/i,12"
    s = Split(s, ",")
    For i = 0 To UBound(s)
        msg = msg & "s(" & i & ") = " & s(i) & vbCrLf
    Next

    msg = msg & vbCrLf & "ParseArguments()" & vbCrLf
    s = "X,14,Sin(12)*56/i,12"
    s = ParseArguments(s)
    For i = 0 To UBound(s)
        msg = msg & "s(" & i & ") = " & s(i) & vbCrLf
    Next

    MsgBox msg

    i tested out and both return same result, or is it you are writing an optimized version of microsoft split?

    BC_Programmer

      Topic Starter

      Mastermind
    • Typing is no substitute for thinking.
    • Thanked: 1140
      • Yes
      • Yes
      • BC-Programming.com
    • Certifications: List
    • Computer: Specs
    • Experience: Beginner
    • OS: Windows 11
    Re: Function Call argument list parsing...
    « Reply #3 on: April 23, 2009, 09:33:20 AM »
    ::) ::) ::) ::)

    why dont you use split function?

    Code: [Select]
    msg = "SPLIT" & vbCrLf
    s = "X,14,Sin(12)*56/i,12"
    s = Split(s, ",")
    For i = 0 To UBound(s)
        msg = msg & "s(" & i & ") = " & s(i) & vbCrLf
    Next

    msg = msg & vbCrLf & "ParseArguments()" & vbCrLf
    s = "X,14,Sin(12)*56/i,12"
    s = ParseArguments(s)
    For i = 0 To UBound(s)
        msg = msg & "s(" & i & ") = " & s(i) & vbCrLf
    Next

    MsgBox msg

    i tested out and both return same result, or is it you are writing an optimized version of microsoft split?


    I'm not using split because it doesn't work.


    tell me what the string "34,Sin(1),RANDOM(1,4)" returns.

    the desired output is {"34","Sin(1)","RANDOM(1,4)"}, but split decides otherwise.

    the reason is of course that it cares not for brackets, so I keep track of the number of open brackets as I proceed through, and also the number of quotes (again- Split cares not for quotes)


    As usual, I have no idea of what you are doing. I do better with 'top down' programming. Perhaps you are using queue/stack solution.
    This item: "Sin(12)*56/i"
    would suggest that yup need an expression evaluator and maybe a floating pint match kit. I will comment on the expression evaluator. This kind of expression is often called a left to right evaluation, but with FORTRAN rules. Which means that he parser is cgoing to to do some look ahead or even read the thing right to left.
    But going going left to right, we parse SIM and determine that is a function we know, will put a token, perhaps a pointer, into a queue (read stack) to identify that item. Next we look inside the left _( and find a constant and next  find the right_) Now can call  SIM with the value. Next find an atom the denotes multiply, so will queue that with the value return by SIM. We find a constant and an atom for divide, lower precedence, so perform what we now have in the queue. Note that divide is coming. Queue it. Next is looks like a variable, go get its value... and so one.

    Anyway. I would use an expression evaluator made by somebody else But this is not new stuff. Was that not done bu Egyptians on clay tablets?
    Sorry.. I  can not take anymore... I need a Tylenol.




    My parser was originally a Stack-based solution but upon reflection it really ends up as a recursive descent parser.

    Support for complex numbers and a number of interfaces one can implements plugins through (Ioperable objects that support operations/functions on their instances), and "IEvalEvents" implementors that can add new Functions/Operators to the evaluator as well. The Stack created initially to store the parsed expression is really just a list of tokens (functions, operators, literals) which have all the data they need. The Function and SubExpression types (IT_FUNCTION and IT_SUBEXPRESSION) are most interesting, because they perform no real parsing of the arguments/contents themselves. The SUBEXPRESSION type (with parentheses) simply creates another parser object, setting it's expression to the contents of the paretheses. Same for the Function parameters (Except the function parameters store a number of "CParser" objects in the Linked List that "RipFormula" creates.


    Because of this, it can Optimize far better. The parser stores each Stack created into a global collection, keyed by the expression used to build it. It consults this collection to determine wether another parser object hasn't already parsed it. If it has, it simply retrieves the stack as stored; otherwise, it rips the formula itself, and then stores the resulting stack.


    executing, for example:

    Code: [Select]
    "Sin(RANDOM(1,10*STORE(X,5))*{1,2,3}"

    will end up in a grand total of 9 stacks being cached. This means that if, for example, a later execution of:

    [code]
    10*STORE(X,5)
    is performed, anywhere- there will be no need to perform the expensive task of ripping the stack from the string, instead it will just grab it from the stack.

    Of course, storing all these stacks can me hard on RAM with longer runs, so each "configuration set" (a name given to a set of Plugins and settings; useful for applications that always want to have the parser load plugins specific to the app) each one can specify the maximum number of Cached Parse Stacks. the Purges are performed by deleting old entries from the collection until the count is acceptable.

    Additionally, it has intrinsic support for method calls on object-type variables. For example, the following would start word and display it, if it's installed:

    Code: [Select]
    STORE(Word,CREATEOBJECT("Word.Application");
    Word@VISIBLE(True);
    STORE(Word,Nothing);


    the @ operator implements eventually) a set of queries to the objects type information and then performs InvokeHook with the appropriate parameters, after inspecting wether the member is a method or property.

    In any case, I had to fix a few subtle issues with the previous incarnation:


    Code: [Select]
    Public Function ParseArguments(ByVal FromString As String, Optional ByVal BracketStart As String = DefaultBracketStart, _
    Optional ByVal Bracketend As String = DefaultBracketEnd, Optional ByVal ARGUMENTSEP As String = ARGSEP, Optional ByRef startpos As Long = 1) As String()

        'NOTE: this routine REALLY needs to be rewritten!
        'full of hacks and kludges...

        'ParseArguments Function:
        'assumes the given string is a parameter lists.
        'keeping track of parentheses and quotes, it finds the next ARGSEP that is not part of some other token.
       
        'Added May 06 2008 10:50 PM:
       
        'recognize the use of a "To" keyword between two arguments.
       
        Dim CurrPos As Long
        Dim SplMake() As String
        Dim CurrArgument As Long, Char As String
        Dim inquote As Boolean, BracketCount As Long
        Dim ArgStart As Long
        'If there isn't even a separator in the string, at all, return the string as the single argument
        If InStr(FromString, ARGUMENTSEP) = 0 Then
            ReDim SplMake(0)
            SplMake(0) = FromString
            ParseArguments = SplMake
        Exit Function
       
        End If
       
       
        If Len(FromString) <= 0 Then
            Erase SplMake()
           
            ParseArguments = SplMake
            Exit Function
        End If
       
       
       
        CurrArgument = 0
        CurrPos = startpos
        ArgStart = CurrPos
        Do
            Char = Mid$(FromString, CurrPos, 1)
           
            Select Case True
            Case Char = """"
                inquote = Not inquote
       
            Case inquote And CurrPos < Len(FromString)
                'IGNORE!>
                'ignore as long as we are in a string.
               
            Case InStr(BracketStart, Char) <> 0 And Char <> ""
                BracketCount = BracketCount + 1
            Case InStr(Bracketend, Char) <> 0 And Char <> ""
           
                BracketCount = BracketCount - 1
                   If BracketCount < 0 Then
                       'Sigh.
                       BracketCount = 0
                       
                   End If
                   
             
            Case InStr(CurrPos, FromString, ARGUMENTSEP) = 0 And BracketCount = 0 And Trim$(Mid$(FromString, CurrPos)) <> ""
               
                ReDim Preserve SplMake(CurrArgument)
                SplMake(CurrArgument) = Mid$(FromString, CurrPos)
                Exit Do
            Case Char = ARGUMENTSEP, (CurrPos >= Len(FromString))
                'Stop
                'If ((Not Inquote) Or _
                 (CurrPos >= Len(FromString))) And BracketCount = 0 Then
                 If ((Not inquote) And BracketCount = 0) Or (CurrPos >= Len(FromString)) Then
                    'wow, that is very strange, heh?
                    'This will only be true when inquote=false (0) and bracketcount=0 as well.
                    ReDim Preserve SplMake(CurrArgument)
                    SplMake(CurrArgument) = Mid$(FromString, ArgStart, Abs(CurrPos - ArgStart))
                    If right$(SplMake(CurrArgument), 1) = ARGUMENTSEP Then
                        SplMake(CurrArgument) = Mid$(SplMake(CurrArgument), 1, Len(SplMake(CurrArgument)) - 1)
                    End If
                    CurrArgument = CurrArgument + 1
                    ArgStart = CurrPos + 1
                   
                    If CurrPos >= Len(FromString) Then
    '                    If InStr(1, Bracketend, right$(SplMake(CurrArgument - 1), 1), vbTextCompare) <> 0 And BracketCount <= 0 Then
    '                        SplMake(CurrArgument - 1) = Mid$(SplMake(CurrArgument - 1), 1, Len(SplMake(CurrArgument - 1)) - 1)
    '                    End If
                       
                   
                        Exit Do
                       
                    End If
                End If
           
            End Select
       
            CurrPos = CurrPos + 1
        Loop
        If UBound(SplMake) = 0 Then
            If SplMake(0) = "" Then
                'there weren't any arguments- return an actual "Empty" array to denote this to the caller.
                Erase SplMake
            End If
        End If
        startpos = CurrPos
        ParseArguments = SplMake
    End Function

    The core itself implements support for Complex arithmetic, (heh, all I had to do was create an "IOperable" implementing object for Complex numbers, handle the appropriate interface functions and add a variable "i" to a complex number with it's imaginary part set to 1 and the realpart set to 0, and then set that variable as read-only.),Arrays (for example, {1,2,3}*{5,6,7} will work fine).


    quite an interesting library.
    [/code]
    I was trying to dereference Null Pointers before it was cool.

    Geek-9pm


      Mastermind
    • Geek After Dark
    • Thanked: 1026
      • Gekk9pm bnlog
    • Certifications: List
    • Computer: Specs
    • Experience: Expert
    • OS: Windows 10
    Re: Function Call argument list parsing...
    « Reply #4 on: April 23, 2009, 11:13:15 AM »
    Quote
    from BC programmer
    The core itself implements support for Complex arithmetic, (heh, all I had to do was create an "IOperable" implementing object for Complex numbers, handle the appropriate interface functions and add a variable "i" to a complex number with it's imaginary part set to 1 and the realpart set to 0, and then set that variable as read-only.),Arrays (for example, {1,2,3}*{5,6,7} will work fine).

    Did you really do that on stone tablets?
    So, complex numbers is not a FORTRAN thing?
    NO, IT WAS JUST NOW INVENTED BY INTEL l!
    http://software.intel.com/en-us/articles/using-sse3-technology-in-algorithms-with-complex-arithmetic-1/

    AAahh. The Tylenol was not enough,
    and I just ran out of Advil.



    BC_Programmer

      Topic Starter

      Mastermind
    • Typing is no substitute for thinking.
    • Thanked: 1140
      • Yes
      • Yes
      • BC-Programming.com
    • Certifications: List
    • Computer: Specs
    • Experience: Beginner
    • OS: Windows 11
    Re: Function Call argument list parsing...
    « Reply #5 on: April 24, 2009, 09:10:08 AM »
    I realize fortran implemented complex numbers as part of the core.


    Name one Expression evaluator, that I can get for FREE that can do half of what Mine is capable of and you get- well, nothing, I guess. It's the thought that counts.


    Also, Complex numbers existed in mathematics long before fortran or any computers existed.
    I was trying to dereference Null Pointers before it was cool.

    Geek-9pm


      Mastermind
    • Geek After Dark
    • Thanked: 1026
      • Gekk9pm bnlog
    • Certifications: List
    • Computer: Specs
    • Experience: Expert
    • OS: Windows 10
    Re: Function Call argument list parsing...
    « Reply #6 on: April 24, 2009, 04:14:25 PM »
    BC programmer, you need some FORTRAN. Now every day, but once a month may be enough. Just this year the FREE source code for FORTRAN 77 was released.
    http://www.thefreecountry.com/compilers/fortran.shtml
    It this is of any value, let me know. I am curios to see how they did the parser way back then. The say there was a FORTRAN interpreter. Huh? I was taught the FORTRAN is NEVER done that way. Did they deceive me?