Get HTML Source Code 1

[Omnipresence]

Hello,

I am trying to convert an application from VB 6.0 to VB .Net and ran into a problem. I have a function that uses MSXML. My .Net application now requires a DLL file in order for it to work. I would like my program to be self contained within the executable. Can someone assist me in converting the following function to VB .Net?

--
PublicFunction GetPageHTML(ByVal URL AsString) AsString
Dim HttpReq AsNew MSXML2.XMLHTTP30
Try
HttpReq.open("GET", URL, False)
HttpReq.send()
GetPageHTML = HttpReq.responseText()
HttpReq = Nothing
Catch ex As Exception
Return ex.Message
EndTry
EndFunction
--

What I'm using the function for is to grab the HTML from a work intranet page (an https:// page). The webpage is a large table displaying numeric values. It's all generated from coldfusion. I believe the webpage uses windows authentication. When you load the page into a browser, you are given a pop-up asking for a login/password; then it doesn't ask again unless you close the browser.

The code above duplicated this as my program, upon first being launched, forces the pop-up box. After that, the pop-up box never comes up again unless the program is closed and re-launched. My program parses the HTML displaying only a few of the values from the webpage. It will auto-update every 5 minutes or you can force an update by clicking on the "update" button.

I have tried using system.net.webrequest but I receive a (401)Unauthorized exception error when I use the .getresponse. (I'm even trying with credentials but I'm probably doing it wrong). I would prefer the windows authentication pop-up box to come up so users can just sign into that but I will code a login page on my form if I have to hard code the login/password.

Any help is greatly appreciated.

Thanks,
Omnipresence

 

[AnonGod]

Here's a couple of lines of code I've been playing with. Can't get it to work over HTTPS though.
See if this helps you out any. Good stuff in the WebClient class.
-Andrew

In case I left out anything, do an Imports System.Net.Webclient and Imports System.Text (for ascii conversion I think) and Imports System.IO (for streamreader?)

Sub webclient5()
    Dim wc As New System.Net.WebClient
    Dim uri As String = "[URL unfurl="true"]http://www.yahoo.com"[/URL]
    Dim myStream As Stream = wc.OpenRead(uri) 'Returns Response body (HTML)
    Dim sr As New StreamReader(myStream)

    Dim htmlbody as string = sr.ReadToEnd
    End Sub

Sub webclient3()
    Dim wc As New System.Net.WebClient
    Dim pageData As [Byte]() = wc.DownloadData("[URL unfurl="true"]http://www.yahoo.com")[/URL]
    Dim pageHtml As String = System.Text.Encoding.ASCII.GetString(pageData)
    End Sub

GL, if you have any ideas on getting/sending data over https, I'd love it.

 

[AnonGod]

On further review, I got this https to work, but I do not have an environment to test a login.
Hope this helps.
... now if I can just get posting data to work over https...
-Andrew

Imports System.Net
...
Private Sub Whatever()
    Dim myRequest As WebRequest = WebRequest.Create("[URL unfurl="true"]https://www4.usbank.com/internetBanking/[/URL]
RequestRouter?requestCmdId=DisplayLoginPage")
    Dim myResponse As WebResponse = myRequest.GetResponse
    Dim sr As New System.IO.StreamReader(myResponse.GetResponseStream)

    Dim ResponseString as String
    ResponseString = myResponse.Headers.ToString & vbCrLf
    ResponseString &= sr.ReadToEnd

    myResponse.Close()
End Sub

 

[Omnipresence]

Thanks AnonGod.

This is what I initially tried to do with login/password. As far as I can tell, you need to use credentials with webrequest. Only problem is when I send the getresponse, I get back a (401)Unauthorized error. I'm not sure if this is a client or server issue. I was hoping to avoid using credentials as I can't see to get them to work but it appears that might be the only way to go. I do appreciate your response.

I can get this to work without credentials on other web pages so I know everything but the credentials part works.

Thanks,
Omnipresence