INTELLIGENT WORK FORUMS
FOR COMPUTER PROFESSIONALS

Log In

Come Join Us!

Are you a
Computer / IT professional?
Join Tek-Tips Forums!
  • Talk With Other Members
  • Be Notified Of Responses
    To Your Posts
  • Keyword Search
  • One-Click Access To Your
    Favorite Forums
  • Automated Signatures
    On Your Posts
  • Best Of All, It's Free!

*Tek-Tips's functionality depends on members receiving e-mail. By joining you are opting in to receive e-mail.

Posting Guidelines

Promoting, selling, recruiting, coursework and thesis posting is forbidden.

Jobs

Internet

How To Programmatically Fill In a Form (HTTP GET/POST) by wgcs
Posted: 28 Oct 03 (Edited 1 Sep 05)

(please note: the tek-tips parser has changed some of the simple ampersand characters in the following to being the encoded & ... so anywhere you see & imagine it just has & ;)
Forms on the web can be submitted in two different ways: GET and POST.  (HTTP describes the protocol 'suite' for the web, "HyperText Transfer Protocol" )

HTTP GET -- Simply includes the fields in the form as part of the request URL.  Basically takes the URL from the "ACTION" parameter of the "FORM" tag, adds "?" to seperate the URL from the field values, then adds all the field values as "Name=value" pairs, pairs seperated by "&".  ie: the Google search form using url http://www.google.com/search then adds "?" followed by the fields of the form: hl, ie, oe, q and btnG, like this:
"hl=en&ie=UTF-8&oe=UTF-8&q=search+terms+here&btnG=Google+Search"
Of course, the characters "?", "=" and "&" would be confusing if they appeared in the field values, so the field values should be URL-Encoded.  The final google GET url is: http://www.google.com/search?hl=en&ie=UTF-8&oe=UTF-8&q=search+terms+here&btnG=Google+Search

HTTP POST -- POST, on the other hand, does not put the parameters in the request URL.  Instead, after connecting to the server and issuing a "POST" command to the specified FORM ACTION URL, the browser sends the form values in the request header.  Since there is a limitation on the length of a URL, POST is a way to send larger amounts of information (like entire files) to a form processor on a web server.  Many web forms use POST even when only accepting a couple form fields: POST prevents the user from bookmarking the result of a form submission.

The code here is primarily concerned with POSTing form data, since GET is so easy.

There are many third party utilities to help with POSTing data, and some tools from Microsoft, too, that may or may not be installed in your users' Windows installs, so I'll start with a very basic, direct to TCP/IP WinSockets illustration of what goes on with a POST:
( The GET and SendMail support here is a bonus: I keep them together in this utility class )

CODE

DEFINE CLASS SocketWrapper As Custom
  * Author: William GC Steinford
  * Based on Anatoliy Mogylevets SMTP code at http://fox.wikis.com/wc.dll?Wiki~SendSmtpEmail
  #DEFINE SMTP_PORT    25
  #DEFINE HTTP_PORT    80
  #DEFINE AF_INET       2
  #DEFINE SOCK_STREAM   1
  #DEFINE IPPROTO_TCP   6
  #DEFINE SOCKET_ERROR -1
  #DEFINE FD_READ       1

  #DEFINE CRLF          chr(13)+chr(10)
  
    host     = ""
    IP       = ""
    Port     = 80
    hSocket  = 0
    cIn      = ""
    WaitForRead = 0
    UserAgent      = 'Mozilla/4.0'
    AcceptLanguage = 'en-us'

  PROCEDURE Init()
      THIS.decl
    IF WSAStartup(0x202, Repli(Chr(0),512)) <> 0
    * unable to initialize Winsock on this computer
        RETURN .F.
    ENDIF
    RETURN .T.
  ENDPROC

  PROCEDURE Destroy
    = WSACleanup()
  ENDPROC
  
  PROCEDURE Host_Assign( vNewVal )
    if empty(vNewVal)
      THIS.IP = ""
    else
      THIS.IP = THIS.GetIP(vNewVal)
    endif
    if not empty(THIS.IP)
      THIS.Host = vNewVal
    else
      THIS.Host = ""
    endif
  ENDPROC

  PROTECTED FUNCTION GetIP( pcHost )
  #DEFINE HOSTENT_SIZE 16
      LOCAL nStruct, nSize, cBuffer, nAddr, cIP
      nStruct = gethostbyname(pcHost)
      IF nStruct = 0
          RETURN ""
      ENDIF
      cBuffer = Repli(Chr(0), HOSTENT_SIZE)
      cIP = Repli(Chr(0), 4)
      = CopyMemory(@cBuffer, nStruct, HOSTENT_SIZE)
      = CopyMemory(@cIP, THIS.buf2dword(SUBS(cBuffer,13,4)),4)
      = CopyMemory(@cIP, THIS.buf2dword(cIP),4)
  RETURN inet_ntoa(THIS.buf2dword(cIP))
  ENDFUNC

  PROTECTED FUNCTION Connect
    LOCAL cBuffer, cPort, cHost, lResult
    THIS.hSocket = socket(AF_INET, SOCK_STREAM, IPPROTO_TCP)
    IF THIS.hSocket = SOCKET_ERROR
        RETURN .F.
    ENDIF
    
    cPort = THIS.num2word(htons(THIS.Port))
    nHost = inet_addr(THIS.IP)
    cHost = THIS.num2dword(nHost)
    cBuffer = THIS.num2word(AF_INET) + cPort + cHost + Repli(Chr(0),8)
    lResult = (ws_connect(THIS.hSocket, @cBuffer, Len(cBuffer))=0)
  RETURN lResult

  FUNCTION httpGet( pcServer, pcUrl )
    LOCAL lResult
    THIS.Host = pcServer
    IF THIS.Connect()
        THIS.snd('GET '+pcURL+' http/1.0'+crlf)
        THIS.snd('Accept: */*'+crlf)
        THIS.snd('Accept-Language: '+THIS.AcceptLanguage+crlf)
        THIS.snd('Accept-Encoding: gzip, deflate'+crlf)
        THIS.snd('User-Agent: '+THIS.UserAgent+crlf)
        THIS.snd('Host: '+pcServer+crlf)
        THIS.snd('Pragma: no-cache'+crlf)
        THIS.snd(crlf,.t.) && End of headers
        *info=url encoded string
        lResult = .T.
    ELSE
        lResult = .F.
    ENDIF
    THIS.Disconnect()
  ENDFUNC
  
  FUNCTION httpPost( pcServer, pcUrl, pcData, poFdbk )
The parameters are:

httpPost( pcServer, pcUrl, pcData, poFdbk )
* Meaning of parameters:
* pcServer: IP or hostname of the server to POST to
*
* pcURL: the URL of the page (CGI/Servlet/PHP, whatever)
*        on the server that the POST data will be sent to.
*
* pcData: the url-formatted string comprised of
*         name=value&name2=value2 data,
*         where all "value" data is UrlEncoded
*
* poFdBk: (optional) A reference to an object that has
*         a method called "Feedback" which accepts
*         a single decimal value, between 0 and 1,
*         which indicates the portion of the transfer
*         that is complete.  
*         I usually add a "Feedback" method to a form,
*         and pass THISFORM as poFdbk, and in
*         THISFORM.Feedback, just update a progressbar
*         or something.

    LOCAL lResult, lnLen, lnComplete, lcRemain, lcSnd
    THIS.Host = pcServer
    IF THIS.Connect()
        THIS.snd('POST '+pcURL+' http/1.0'+crlf)
        THIS.snd('Content-Type:application/x-www-form-urlencoded'+crlf)
        THIS.snd('Accept: */*'+crlf)
        THIS.snd('Accept-Language: '+THIS.AcceptLanguage+crlf)
        THIS.snd('Accept-Encoding: gzip, deflate'+crlf)
        THIS.snd('User-Agent: '+THIS.UserAgent+crlf)
        THIS.snd('Host: '+pcServer+crlf)
        lnLen = len(pcData)
        THIS.snd('Content-Length: '+tran(lnLen)+crlf)
        THIS.snd('Pragma: no-cache'+crlf)
        THIS.snd(crlf) && End of headers
        * If we have a valid feedback object, provide feedback
        if vartype(poFdbk)='O' and PEMStatus(poFdbk,'Feedback',5) ;
           and upper(PEMStatus(poFdbk,'Feedback',3))='METHOD'
          lcRemain   = pcData
          lnComplete = 0
          poFdbk.Feedback( 0 )
          do while len(lcRemain)>0
            lcSnd      = LEFT( lcRemain, 100 )
            lcRemain   = SUBSTR( lcRemain, 101 )
            THIS.snd(lcSnd)
            lnComplete = lnComplete+len(lcSnd)
            poFdbk.Feedback( lnComplete/lnLen*100 )
          enddo
          THIS.snd("",.t.) && get a response, too.
        else && no feedback object, just send one big chunk.
          THIS.snd(pcData,.t.) && get a response, too.
        endif
        lResult = .T.
    ELSE
        lResult = .F.
    ENDIF
    THIS.Disconnect()
  ENDFUNC
  
  FUNCTION SendMail( pcSender, pcRecipient, pcSubject, pcBody )
    LOCAL lResult
    IF THIS.Connect()
        THIS.snd('HELO', .T.)
        THIS.snd('MAIL FROM:<' + pcSender + '>', .T.)
        THIS.snd('RCPT TO:<' + pcRecipient + '>', .T.)
        THIS.snd('DATA', .T.)
        THIS.snd('From: ' + pcSender)
        THIS.snd('To: ' + pcRecipient)
        THIS.snd('Subject: ' + pcSubject)
        THIS.snd("")
        THIS.snd(pcBody)
        THIS.snd('.', .T.)
        THIS.snd('QUIT', .T.)
        lResult = .T.
    ELSE
        = MessageB('Unable to connect to [' + THIS.Host +;
            '] on port ' + LTRIM(STR(SMTP_PORT)) + '. ',;
            48, ' Connection error')
        lResult = .F.
    ENDIF
    THIS.Disconnect()
  RETURN lResult
  ENDFUNC

  Function URLencode
  LPARAMETER pcInStr
  *  ' encode Percent signs
  *  '        Double Quotes
  *  '        CarriageReturn / LineFeeds
  
  LOCAL lcOut, lnI
    * StrTran is WAY faster than building the string in memory
    lcOut = StrTran(pcInStr, [%], '%25' )
    lcOut = StrTran(lcOut,   [+], '%2B' )
    lcOut = StrTran(lcOut,   [ ], '+'   )
    for lnI = 0 to 31
      lcOut = StrTran( lcOut, chr(lnI), '%' + Right( Transform(lnI,'@0'), 2 ) )
    endfor
    for lnI = 127 to 255
      lcOut = StrTran( lcOut, chr(lnI), '%' + Right( Transform(lnI,'@0'), 2 ) )
    endfor

    RETURN lcOut  
  ENDFUNC && UrlEncode
  
  FUNCTION Disconnect
    if THIS.hSocket<>SOCKET_ERROR
      = closesocket(THIS.hSocket)
    endif
    THIS.hSocket = SOCKET_ERROR
  ENDFUNC

  FUNCTION snd(cData, lResponse)
    LOCAL cBuffer, nResult, cResponse
    cBuffer = cData && + CrLf
    nResult = send(THIS.hSocket, @cBuffer, Len(cBuffer), 0)
    IF nResult = SOCKET_ERROR
        RETURN .F.
    ENDIF
    IF Not lResponse
        RETURN .T.
    ENDIF

    LOCAL hEventRead, nWait, cRead
    DO WHILE .T.
        * creating event, linking it to the socket and wait
        hEventRead = WSACreateEvent()
        = WSAEventSelect(THIS.hSocket, hEventRead, FD_READ)

        * 1000 milliseconds can be not enough
        THIS.WaitForRead = WSAWaitForMultipleEvents(1, @hEventRead, 0, 2000, 0)
        = WSACloseEvent(hEventRead)

        IF THIS.WaitForRead <> 0 && error or timeout
            EXIT
        ENDIF
        
        * reading data from connected socket
        THIS.cIn = THIS.cIn+THIS.Rd()
    ENDDO
  RETURN .T.
  ENDFUNC

  PROTECTED FUNCTION Rd
  #DEFINE READ_SIZE 16384
    LOCAL cRecv, nRecv, nFlags
    cRecv = Repli(Chr(0), READ_SIZE)
    nFlags = 0
    nRecv = recv(THIS.hSocket, @cRecv, READ_SIZE, nFlags)
    RETURN Iif(nRecv<=0, "", LEFT(cRecv, nRecv))
  ENDFUNC

  PROCEDURE decl
    DECLARE INTEGER gethostbyname IN ws2_32 STRING host
    DECLARE STRING inet_ntoa IN ws2_32 INTEGER in_addr
    DECLARE INTEGER socket IN ws2_32 INTEGER af, INTEGER tp, INTEGER pt
    DECLARE INTEGER closesocket IN ws2_32 INTEGER s
    DECLARE INTEGER WSACreateEvent IN ws2_32
    DECLARE INTEGER WSACloseEvent IN ws2_32 INTEGER hEvent
    DECLARE GetSystemTime IN kernel32 STRING @lpSystemTime
    DECLARE INTEGER inet_addr IN ws2_32 STRING cp
    DECLARE INTEGER htons IN ws2_32 INTEGER hostshort
    DECLARE INTEGER WSAStartup IN ws2_32 INTEGER wVerRq, STRING lpWSAData
    DECLARE INTEGER WSACleanup IN ws2_32

    DECLARE INTEGER connect IN ws2_32 AS ws_connect ;
        INTEGER s, STRING @sname, INTEGER namelen

    DECLARE INTEGER send IN ws2_32;
        INTEGER s, STRING @buf, INTEGER buflen, INTEGER flags

    DECLARE INTEGER recv IN ws2_32;
        INTEGER s, STRING @buf, INTEGER buflen, INTEGER flags

    DECLARE INTEGER WSAEventSelect IN ws2_32;
        INTEGER s, INTEGER hEventObject, INTEGER lNetworkEvents

    DECLARE INTEGER WSAWaitForMultipleEvents IN ws2_32;
        INTEGER cEvents, INTEGER @lphEvents, INTEGER fWaitAll,;
        INTEGER dwTimeout, INTEGER fAlertable

    DECLARE RtlMoveMemory IN kernel32 As CopyMemory;
        STRING @Dest, INTEGER Src, INTEGER nLength
  ENDPROC

  FUNCTION buf2dword(lcBuffer)
    RETURN Asc(SUBSTR(lcBuffer, 1,1)) + ;
        BitLShift(Asc(SUBSTR(lcBuffer, 2,1)), 8) +;
        BitLShift(Asc(SUBSTR(lcBuffer, 3,1)), 16) +;
        BitLShift(Asc(SUBSTR(lcBuffer, 4,1)), 24)
  ENDFUNC
  
  FUNCTION num2dword(lnValue)
  #DEFINE m0 256
  #DEFINE m1 65536
  #DEFINE m2 16777216
      IF lnValue < 0
          lnValue = 0x100000000 + lnValue
      ENDIF
      LOCAL b0, b1, b2, b3
      b3 = Int(lnValue/m2)
      b2 = Int((lnValue - b3*m2)/m1)
      b1 = Int((lnValue - b3*m2 - b2*m1)/m0)
      b0 = Mod(lnValue, m0)
  RETURN Chr(b0)+Chr(b1)+Chr(b2)+Chr(b3)
  ENDFUNC
  
  FUNCTION num2word(lnValue)
    RETURN Chr(MOD(m.lnValue,256)) + CHR(INT(m.lnValue/256))
  ENDFUNC
ENDDEFINE


If you need to support authentication, or more features than the above code has, either write them yourself (and, I'd appreciate if you'd allow me to include your improvements here), or (particularly if you don't want to be responsible for complex features working) use an external tool.  Here are some you may explore:

WinHttp control:  http://msdn.microsoft.com/library/default.asp?url=/library/en-us/winhttp/http/about_winhttp.asp
( MS control included in:
WinHTTP 5.1 is available only with Windows Server 2003, Windows XP SP1, and Windows 2000 SP3. A redistributable merge module (.msm) file is not available for WinHTTP 5.1.
The WinHTTP 5.0 redistributable, winhttp5.dll, runs on Windows 2000 or later, or Microsoft Windows NT 4.0 with Microsoft Internet Explorer 5.01 or later.
Important  With the release of WinHTTP Version 5.1, the WinHTTP 5.0 download is no longer available.

Reference: http://msdn.microsoft.com/library/default.asp?url=/library/en-us/winhttp/http/what_s_new_in_winhttp_5_1.asp
http://msdn.microsoft.com/library/default.asp?url=/library/en-us/winhttp/http/winhttp_versions.asp )

CODE

    loHTTP = CREATEOBJECT('WinHttp.WinHttpRequest.5.1')
    * OR:
    loHTTP = CREATEOBJECT('WinHttp.WinHttpRequest.5.0') && or is that just '... .5'???
    loHTTP.Open('POST', 'http://www.SomeSite.cgi'; , .F.)
    loHTTP.SetCredentials('UserName', 'Password', 0)
    loHTTP.SetClientCertificate('CertficateName')
    loHTTP.SetRequestHeader('content-type', 'application/x-www-form-urlencoded')
    loHTTP.Send('SendThis')
( Thanks to Ralph/rkolva in thread184-664165 for this code )

MS XML HTTP request object:
http://msdn.microsoft.com/library/default.asp?url=/library/en-us/xmlsdk30/htm/xmobjxmldomserverxmlhttp.asp
Code excerpted from: http://fox.wikis.com/wc.dll?Wiki~SendHotmailMessageFromFoxPro

CODE

    oHttp = CreateObject('MSXML2.ServerXMLHTTP')
    WITH oHttp
        .Open('POST', cSendmsgHref, 0)
        .SetRequestHeader('Content-Type', 'message/rfc821')
        .SetRequestHeader('SAVEINSENT', 'f') && set field properties
        .Send(cMessage)
        nRespCode = VAL(SUBSTR(.GetResponseHeader('X-Dav-Error'), 1,3))
    ENDWITH

3rd party tool from West Wind:
http://www.west-wind.com/wwIPStuff.asp

Back to Microsoft: Visual FoxPro FAQ Index
Back to Microsoft: Visual FoxPro Forum

My Archive

Resources

Close Box

Join Tek-Tips® Today!

Join your peers on the Internet's largest technical computer professional community.
It's easy to join and it's free.

Here's Why Members Love Tek-Tips Forums:

Register now while it's still free!

Already a member? Close this window and log in.

Join Us             Close