In this example, You will learn hot to crawl a web page or file using a virtual path and store the response locally.
Crawling refers to request the web page data and get the reponse data programmatically. You can easily do this in .net.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 | // Be sure to include these namespaces using System.Net; using System.IO; ... ... WebRequest request = WebRequest.Create("http://www.samplepage.com/"); using (WebResponse response = request.GetResponse()) { using (StreamReader responseReader = new StreamReader(response.GetResponseStream())) { string responseData = responseReader.ReadToEnd(); using (StreamWriter writer = new StreamWriter(@"C:\samplePage.html")) { writer.Write(responseData); } } } |
