平时工作中长期需要用到通过HTTP调用API进行数据获取以及文件上传下载(C#,JAVA...都会用到)。这里获取的是包括网页的所有信息。如果单纯需要某些数据内容。可以自己构造函数甄别抠除出来!一般的做法是根据源码的格式,用正则来过滤出你需要的内容部分。

  C#中的调用:

    方式一:webclient和httpclient

    方式二:WebRequest和webresponse

    方式三:通过js 直接访问api接口,页面通过jquery调用

    方式四:WebBrowser

1.webclient和httpclient

  实例①

#region webclient应用
#region Controllers
MyImageServerEntities db = new MyImageServerEntities();
public ActionResult Index()
{
return View();
}
public ActionResult FileUpload()
{
HttpPostedFileBase file = Request.Files["fileUp"];
string fileName = Path.GetFileName(file.FileName);
string fileExt = Path.GetExtension(fileName);
if (fileExt == ".jpg")
{
var list = db.ImageServerInfo.Where<ImageServerInfo>(u => u.FlgUsable == true).ToList();//找出可用的图片服务器.
int count = list.Count();
Random random = new Random();
int r = random.Next();
int i = r % count;
string serverUrl = list[i].ServerUrl;
int serverId = list[i].ServerId;
string url = "http://" + serverUrl + "/FileUp.ashx?serverId=" + serverId + "&fileExt=" + fileExt;
WebClient webClient = new WebClient();
webClient.UploadData(url, StreamToBytes(file.InputStream)); }
return Content("ok"); }
public ActionResult ShowImage()
{
var list = db.ImageServerInfo.Where<ImageServerInfo>(c => c.FlgUsable == true).ToList();
ViewData["imageServerList"] = list;
return View();
}
private byte[] StreamToBytes(Stream stream)
{
byte[] buffer = new byte[stream.Length];
stream.Read(buffer, , buffer.Length);
stream.Seek(, SeekOrigin.Begin);
return buffer;
}
#endregion #region Model
context.Response.ContentType = "text/plain";
int serverId = int.Parse(context.Request["serverId"]);
string fileExt = context.Request["fileExt"];
string dir = "/ImagePath/" + DateTime.Now.Year + "/" + DateTime.Now.Month + "/" + DateTime.Now.Day + "/";
Directory.CreateDirectory(Path.GetDirectoryName(context.Request.MapPath(dir)));
string newfileName = Guid.NewGuid().ToString();
string fullDir = dir + newfileName + fileExt;
using (FileStream fileStream = File.Open(context.Request.MapPath(fullDir), FileMode.OpenOrCreate))
{
context.Request.InputStream.CopyTo(fileStream);
MyImageServerEntities db = new MyImageServerEntities();
ImageInfo imageInfo = new ImageInfo();
imageInfo.ImageName = fullDir;
imageInfo.ImageServerId = serverId;
db.ImageInfo.Add(imageInfo);
db.SaveChanges();
}
#endregion
#endregion #region httpclient
创建并初始化对象:
client.BaseAddress = new Uri(url);
client.DefaultRequestHeaders.Accept.Add(new MediaTypeWithQualityHeaderValue("application/json")); 读集合:
HttpResponseMessage response = client.GetAsync(url).Result;
var userList = response.Content.ReadAsAsync<IEnumerable<数据类型>>().Result; 根据编号读对象
HttpResponseMessage response1 = client.GetAsync(url).Result;
var userInfo = response1.Content.ReadAsAsync<数据类型>().Result; 增加:
HttpResponseMessage response = client.PostAsJsonAsync("api/userinfo", userInfo).Result;
使用response.IsSuccessStatusCode判断是否成功
使用response.Content.ToString()获取返回值 修改:
HttpResponseMessage response = client.PutAsJsonAsync("api/userinfo/" + userInfo.UserId, userInfo).Result;
使用response.IsSuccessStatusCode判断是否成功
使用response.Content.ToString()获取返回值 删除:
HttpResponseMessage response = client.DeleteAsync("api/userinfo/" + uid).Result;
使用response.IsSuccessStatusCode判断是否成功
使用response.Content.ToString()获取返回值
#endregion

  实例②

WebClient获取网页数据:
//using System.IO;
try
{
WebClient webClient = new WebClient();
webClient.Credentials = CredentialCache.DefaultCredentials;//获取或设置用于向Internet资源的请求进行身份验证的网络凭据
Byte[] pageData = webClient.DownloadData("http://www.360doc.com/content/11/0427/03/1947337_112596569.shtml");
//string pageHtml = Encoding.Default.GetString(pageData); //如果获取网站页面采用的是GB2312,则使用这句
string pageHtml = Encoding.UTF8.GetString(pageData); //如果获取网站页面采用的是UTF-8,则使用这句
using (StreamWriter sw = new StreamWriter("e:\\ouput.txt"))//将获取的内容写入文本
{
htm = sw.ToString();//测试StreamWriter流的输出状态,非必须
sw.Write(pageHtml);
}
}
catch (WebException webEx)
{
Console.W
} 如果为了实际项目的效率考虑,需要考虑在函数中分配一个内存区域。大概写法如下:
//MemoryStream是一个支持储存区为内存的流。
byte[] buffer = new byte[];
 using (MemoryStream memory = new MemoryStream())
{
int index = , sum = ;
while (index * sum < * )
{
index = reader.Read(buffer, , );
if (index > )
{
memory.Write(buffer, , index);
sum += index;
}
}
//网页通常使用utf-8或gb2412进行编码
Encoding.GetEncoding("gb2312").GetString(memory.ToArray());
if (string.IsNullOrEmpty(html))
{
return html;
}
else
{
Regex re = new Regex(@"charset=(? charset[/s/S]*?)[ |']");
Match m = re.Match(html.ToLower());
encoding = m.Groups[charset].ToString();
}
if (string.IsNullOrEmpty(encoding) || string.Equals(encoding.ToLower(), "gb2312"))
{
return html;
}
}

  实例③

使用WebClient进行文件下载,这里使用了Task异步方式,也可以改为普通方法或静态方法:
/// <summary>
/// 下载文件
/// </summary>
/// <param name="fileUrl">文件地址</param>
/// <param name="filePath">文件的本地路径</param>
/// <returns>文件在本地的存储路径</returns>
private Task GetFileLocalPath(string fileUrl, string filePath)
{
return Task.Run(() =>
{
try
{
using (var mc = new WebClient())
{
mc.DownloadFile(new Uri(fileUrl), filePath);
}
}
catch (Exception ex)
{
LogHelper.WriteErrorLog("下载文件时出现异常。", ex);
}
});
} /// <summary>
/// 下载文件
/// </summary>
/// <param name="fileUrl">文件地址</param>
/// <param name="filePath">文件的本地路径</param>
/// <returns>文件在本地的存储路径</returns>
private Task DownloadFile(string fileUrl, string filePath)
{
return Task.Run(() =>
{
try
{
using (var webClient = new WebClient())
{
var netStream = webClient.OpenRead(fileUrl);
if (netStream != null)
{
FileStream fstr = new FileStream(filePath, FileMode.OpenOrCreate, FileAccess.Write);
byte[] readbyte = new byte[];
int realReadLen = netStream.Read(readbyte, , readbyte.Length);
while (realReadLen > )
{
fstr.Write(readbyte, , realReadLen);
realReadLen = netStream.Read(readbyte, , readbyte.Length);
Thread.Sleep();
}
netStream.Dispose();
fstr.Flush();
fstr.Close();
}
}
}
catch (Exception ex)
{
LogHelper.WriteErrorLog("下载文件时出现异常。", ex);
}
});
} HttpClient,这个类提供的都是异步方法,下面包含了POST、GET、PUT、DELETE四个方法,还有一个SEND方法稍加改动即可实现:
public static async void AsyncPost(string url, string body, Action<RequestResult> callback, int timeoutSeconds = )
{
var requestResult = new RequestResult();
if (!IsConnectedInternet())
{
requestResult.Message = "网络连接错误,请稍后再试。";
callback?.Invoke(requestResult);
return;
} try
{
using (var client = new HttpClient())
{
client.DefaultRequestHeaders.Add("authorization", "LJQfL1A2oeP2fuEiOHo6");
client.Timeout = new TimeSpan(, , timeoutSeconds);
//byte[] requestBody = Encoding.UTF8.GetBytes(body);
HttpContent content = new StringContent(body);
content.Headers.ContentType = new MediaTypeHeaderValue("application/json");
var response = await client.PostAsync(url, content);
//确保HTTP成功状态值
response.EnsureSuccessStatusCode();
//await异步读取最后的JSON
await response.Content.ReadAsStringAsync().ContinueWith(t =>
{
if (t.IsCompleted)
{
requestResult.IsSuccess = true;
requestResult.Result = t.Result;
callback?.Invoke(requestResult);
}
});
}
}
catch (Exception e)
{
if (e.InnerException != null)
requestResult.Message = e.InnerException.Message;
else if (e.Message.Contains("已取消一个任务"))
requestResult.Message = "连接服务器超时,请重试";
else if (e.Message.Contains(""))
requestResult.Message = "连接服务器404,请重试";
else
requestResult.Message = e.Message;
callback?.Invoke(requestResult);
}
} public static async void AsyncGet(string url, Action<RequestResult> callback, int timeoutSeconds = )
{
var requestResult = new RequestResult();
if (!IsConnectedInternet())
{
requestResult.Message = "网络连接错误,请稍后再试。";
callback?.Invoke(requestResult);
return;
} try
{
using (var client = new HttpClient())
{
client.DefaultRequestHeaders.Add("authorization", "LJQfL1A2oeP2fuEiOHo6");
client.Timeout = new TimeSpan(, , timeoutSeconds);
var response = await client.GetAsync(url);
//确保HTTP成功状态值
response.EnsureSuccessStatusCode();
//await异步读取最后的JSON
await response.Content.ReadAsStringAsync().ContinueWith(t =>
{
if (t.IsCompleted)
{
requestResult.IsSuccess = true;
requestResult.Result = t.Result;
callback?.Invoke(requestResult);
}
});
}
}
catch (Exception e)
{
if (e.InnerException != null)
requestResult.Message = e.InnerException.Message;
else if (e.Message.Contains("已取消一个任务"))
requestResult.Message = "连接服务器超时,请重试";
else if (e.Message.Contains(""))
requestResult.Message = "连接服务器404,请重试";
else
requestResult.Message = e.Message;
callback?.Invoke(requestResult);
}
} public static async void AsyncPut(string url, string body, Action<RequestResult> callback, int timeoutSeconds = )
{
var requestResult = new RequestResult();
if (!IsConnectedInternet())
{
requestResult.Message = "网络连接错误,请稍后再试。";
callback?.Invoke(requestResult);
return;
} try
{
using (var client = new HttpClient())
{
client.DefaultRequestHeaders.Add("authorization", "LJQfL1A2oeP2fuEiOHo6");
client.Timeout = new TimeSpan(, , timeoutSeconds);
HttpContent content = new StringContent(body);
content.Headers.ContentType = new MediaTypeHeaderValue("application/json");
var response = await client.PutAsync(url, content);
//确保HTTP成功状态值
response.EnsureSuccessStatusCode();
//await异步读取最后的JSON
await response.Content.ReadAsStringAsync().ContinueWith(t =>
{
if (t.IsCompleted)
{
requestResult.IsSuccess = true;
requestResult.Result = t.Result;
callback?.Invoke(requestResult);
}
});
}
}
catch (Exception e)
{
if (e.InnerException != null)
requestResult.Message = e.InnerException.Message;
else if (e.Message.Contains("已取消一个任务"))
requestResult.Message = "连接服务器超时,请重试";
else if (e.Message.Contains(""))
requestResult.Message = "连接服务器404,请重试";
else
requestResult.Message = e.Message;
callback?.Invoke(requestResult);
}
} public static async void AsyncDelete(string url, Action<RequestResult> callback, int timeoutSeconds = )
{
var requestResult = new RequestResult();
if (!IsConnectedInternet())
{
requestResult.Message = "网络连接错误,请稍后再试。";
callback?.Invoke(requestResult);
return;
} try
{
using (var client = new HttpClient())
{
client.DefaultRequestHeaders.Add("authorization", "LJQfL1A2oeP2fuEiOHo6");
client.Timeout = new TimeSpan(, , timeoutSeconds);
var response = await client.DeleteAsync(url);
//确保HTTP成功状态值
response.EnsureSuccessStatusCode();
//await异步读取最后的JSON
await response.Content.ReadAsStringAsync().ContinueWith(t =>
{
if (t.IsCompleted)
{
requestResult.IsSuccess = true;
requestResult.Result = t.Result;
callback?.Invoke(requestResult);
}
});
}
}
catch (Exception e)
{
if (e.InnerException != null)
requestResult.Message = e.InnerException.Message;
else if (e.Message.Contains("已取消一个任务"))
requestResult.Message = "连接服务器超时,请重试";
else if (e.Message.Contains(""))
requestResult.Message = "连接服务器404,请重试";
else
requestResult.Message = e.Message;
callback?.Invoke(requestResult);
}
} 上面使用到的RequestResult类:
public class RequestResult : IDisposable
{
public bool IsSuccess { get; set; } public string Result { get; set; } public string Message { get; set; } public RequestResult(bool isSuccess = false, string result = "", string message = "")
{
IsSuccess = isSuccess;
Result = result;
Message = message;
} ~RequestResult()
{
Dispose();
} public void Dispose()
{
Dispose(true);
GC.SuppressFinalize(this);//不需要再调用本对象的Finalize方法
} protected virtual void Dispose(Boolean disposing)
{
if (disposing)
{
//--- 清理托管资源 ---//
} //--- 清理非托管资源 ---//
}
}

2.WebRequest和webresponse

  实例①

   public void GetHtml()
{
var url = "http://www.360doc.com/content/11/0427/03/1947337_112596569.shtml";
string strBuff = "";//定义文本字符串,用来保存下载的html
int byteRead = ; HttpWebRequest webRequest = (HttpWebRequest)WebRequest.Create(url);
HttpWebResponse webResponse = (HttpWebResponse)webRequest.GetResponse();
//若成功取得网页的内容,则以System.IO.Stream形式返回,若失败则产生ProtoclViolationException错 误。在此正确的做法应将以下的代码放到一个try块中处理。这里简单处理
Stream reader = webResponse.GetResponseStream();
///返回的内容是Stream形式的,所以可以利用StreamReader类获取GetResponseStream的内容,并以StreamReader类的Read方法依次读取网页源程序代码每一行的内容,直至行尾(读取的编码格式:UTF8)
StreamReader respStreamReader = new StreamReader(reader,Encoding.UTF8); ///分段,分批次获取网页源码
char[] cbuffer = new char[];
byteRead = respStreamReader.Read(cbuffer,,); while (byteRead != )
{
string strResp = new string(char,,byteRead);
strBuff = strBuff + strResp;
byteRead = respStreamReader.Read(cbuffer,,);
}
using (StreamWriter sw = new StreamWriter("e:\\ouput.txt"))//将获取的内容写入文本
{
htm = sw.ToString();//测试StreamWriter流的输出状态,非必须
sw.Write(strBuff);
}
}

  实例②

WebRequest的方式(该方式是post请求的时候传入json字符串转换成流的形式,无需再将json转换成需要的实体对象了),所以这种方式只需写一个公共的方法,每次调用的时候传入地址和参数即可
公共方法如下:
public class WebRequestHelper
{
public static string GetRequest(string url)
{
string responseStr = string.Empty; WebRequest request = WebRequest.Create(url);
request.Method = "Get";
var response = request.GetResponse();
Stream ReceiveStream = response.GetResponseStream();
using (StreamReader stream = new StreamReader(ReceiveStream, Encoding.UTF8))
{
responseStr = stream.ReadToEnd();
} return responseStr;
} public static string PostRequest(string url, string postParameter)
{
string responseStr = string.Empty; WebRequest request = WebRequest.Create(url);
request.Method = "Post";
request.ContentType = "application/json"; byte[] requestData = System.Text.Encoding.UTF8.GetBytes(postParameter);
request.ContentLength = requestData.Length; Stream newStream = request.GetRequestStream();
newStream.Write(requestData, , requestData.Length);
newStream.Close(); var response = request.GetResponse();
Stream ReceiveStream = response.GetResponseStream();
using (StreamReader stream = new StreamReader(ReceiveStream, Encoding.UTF8))
{
responseStr = stream.ReadToEnd();
} return responseStr;
} /// <summary>
/// Http的 WebRequest的特定实现
/// </summary>
/// <param name="Url"></param>
/// <param name="JSONData"></param>
/// <returns></returns>
public static string GetResponseData(string Url, string JSONData)
{
byte[] bytes = Encoding.UTF8.GetBytes(JSONData);
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(Url);
request.Method = "POST";
request.ContentLength = bytes.Length;
request.ContentType = "application/json";
Stream reqstream = request.GetRequestStream();
reqstream.Write(bytes, , bytes.Length); //声明一个HttpWebRequest请求
request.Timeout = ;
//设置连接超时时间
request.Headers.Set("Pragma", "no-cache");
HttpWebResponse response = (HttpWebResponse)request.GetResponse();
Stream streamReceive = response.GetResponseStream();
Encoding encoding = Encoding.UTF8; StreamReader streamReader = new StreamReader(streamReceive, encoding);
string strResult = streamReader.ReadToEnd();
streamReceive.Dispose();
streamReader.Dispose(); return strResult;
}
} 调用的使用方法 ()Get请求 public ActionResult GetAll()
{
string result = WebRequestHelper.GetRequest(url + "api/goods/getlist?pageindex=0&pagesize=10");
JavaScriptSerializer Serializer = new JavaScriptSerializer();
var items = Serializer.DeserializeObject(result);
return Json(items);
} ()Post请求 [HttpPost]
public ActionResult AddGood(string paramObj)
{
string result = WebRequestHelper.PostRequest(url + "api/goods/add", paramObj);
JavaScriptSerializer Serializer = new JavaScriptSerializer();
var items = Serializer.DeserializeObject(result);
return Json(items);
}

  实例③

HttpWebRequest:
/// <summary>
/// 向服务器发送Request
/// </summary>
/// <param name="url">字符串</param>
/// <param name="method">枚举类型的方法Get或者Post</param>
/// <param name="body">Post时必须传值</param>
/// <param name="timeoutSeconds">超时时间,单位秒</param>
/// <returns></returns>
public static string Request(string url, MethodEnum method, string body = "", int timeoutSeconds = )
{
if (!IsConnectedInternet())
return "网络连接错误,请稍后再试。"; try
{
GC.Collect();
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(url);
request.Timeout = timeoutSeconds;
request.Method = method.ToString();
//如果是Post的话,则设置body
if (method == MethodEnum.POST)
{
request.ContentType = "application/json";
request.KeepAlive = false;
byte[] requestBody = Encoding.UTF8.GetBytes(body);
request.ContentLength = requestBody.Length; Stream requestStream = request.GetRequestStream();
requestStream.Write(requestBody, , requestBody.Length);
} return Response(request);
}
catch (Exception ex)
{
if (ex.InnerException != null)
return ex.InnerException.Message;
if (ex.Message.Contains("已取消一个任务"))
return "连接服务器超时,请重试";
if (ex.Message.Contains(""))
return "连接服务器404,请重试";
return ex.Message;
}
} HttpWebResponse:
/// <summary>
/// 返回Response数据
/// </summary>
/// <param name="request"></param>
/// <returns></returns>
private static string Response(HttpWebRequest request)
{
HttpWebResponse response = (HttpWebResponse)request.GetResponse(); string jsonRead = ""; if (response.StatusCode != HttpStatusCode.OK)
{
return response.StatusCode.ToString();
}
//接收过程
if (response.GetResponseStream() != null)
{
StreamReader myStreamReader = new StreamReader(response.GetResponseStream() ?? Stream.Null, Encoding.UTF8);
jsonRead = myStreamReader.ReadToEnd();
myStreamReader.Close();
}
response.Close();
request.Abort(); return jsonRead;
} 上面两个方法需要配合使用,皆为同步方式。当然也可以将上面两个方法合并到一个方法中,可以参见接下来这个方法。 另外是使用HttpWebRequest和HttpWebResponse进行文件上传,采用同步方法异步回调方式: public static void UploadFile(string url, string filePath, string fileName, Action<string> callback)
{
// 时间戳,用做boundary
string timeStamp = DateTime.Now.Ticks.ToString("x"); //根据uri创建HttpWebRequest对象
HttpWebRequest httpReq = (HttpWebRequest)WebRequest.Create(new Uri(url));
httpReq.Method = "POST";
httpReq.AllowWriteStreamBuffering = false; //对发送的数据不使用缓存
httpReq.Timeout = ; //设置获得响应的超时时间(300秒)
httpReq.ContentType = "multipart/form-data; boundary=" + timeStamp; //文件
FileStream fileStream = new FileStream(filePath, FileMode.Open, FileAccess.Read);
BinaryReader binaryReader = new BinaryReader(fileStream); //头信息
string boundary = "--" + timeStamp;
string dataFormat = boundary + "\r\nContent-Disposition: form-data; name=\"{0}\";filename=\"{1}\"\r\nContent-Type:application/octet-stream\r\n\r\n";
string header = string.Format(dataFormat, "file", Path.GetFileName(filePath));
byte[] postHeaderBytes = Encoding.UTF8.GetBytes(header); //结束边界
byte[] boundaryBytes = Encoding.ASCII.GetBytes("\r\n--" + timeStamp + "--\r\n"); long length = fileStream.Length + postHeaderBytes.Length + boundaryBytes.Length; httpReq.ContentLength = length;//请求内容长度 try
{
//每次上传4k
int bufferLength = ;
byte[] buffer = new byte[bufferLength]; //已上传的字节数
long offset = ;
int size = binaryReader.Read(buffer, , bufferLength);
Stream postStream = httpReq.GetRequestStream(); //发送请求头部消息
postStream.Write(postHeaderBytes, , postHeaderBytes.Length); while (size > )
{
postStream.Write(buffer, , size);
offset += size;
size = binaryReader.Read(buffer, , bufferLength);
} //添加尾部边界
postStream.Write(boundaryBytes, , boundaryBytes.Length);
postStream.Close(); string returnValue = "";
//获取服务器端的响应
using (HttpWebResponse response = (HttpWebResponse)httpReq.GetResponse())
{
Stream receiveStream = response.GetResponseStream();
StreamReader readStream = new StreamReader(receiveStream, Encoding.UTF8);
returnValue = readStream.ReadToEnd();
response.Close();
readStream.Close();
} callback?.Invoke(returnValue);
}
catch (Exception)
{
callback?.Invoke("");
}
finally
{
fileStream.Close();
binaryReader.Close();
}
} 上面还用到一个enum叫MethodEnum,包含GET和POST两个枚举值。
还有一个比较特殊的POST方法:
public static string HttpPostFormData(string url, Dictionary<string, string> dic)
{
try
{
GC.Collect();
// 时间戳,用做boundary
string timeStamp = DateTime.Now.Ticks.ToString("x");
string boundary = "----" + timeStamp; HttpWebRequest request = (HttpWebRequest)WebRequest.Create(url);
request.Method = WebRequestMethods.Http.Post;
request.ContentType = "multipart/form-data; boundary=" + boundary;
request.KeepAlive = true;
request.Timeout = ; var stream = new MemoryStream(); //头信息
string dataFormat = "--" + boundary + "\r\nContent-Disposition: form-data; name=\"{0}\"\r\n\r\n{1}\r\n";
foreach (string key in dic.Keys)
{
string s = string.Format(dataFormat, key, dic[key]);
byte[] data = Encoding.UTF8.GetBytes(s);
stream.Write(data, , data.Length);
} //结束边界
byte[] boundaryBytes = Encoding.ASCII.GetBytes("--" + boundary + "--");
stream.Write(boundaryBytes, , boundaryBytes.Length); request.ContentLength = stream.Length;//请求内容长度 Stream requestStream = request.GetRequestStream(); //写入请求数据
stream.Position = 0L;
stream.CopyTo(requestStream);
stream.Close(); requestStream.Close(); return Response(request);
}
catch (Exception e)
{
return e.Message;
}
}

3.通过js 直接访问api接口,页面通过jquery调用

$.ajax({
type: "get",
url: "/goods/getall",
dataType: "json",
data: null,
//data: JSON.stringify({
// id: "382accff-57b2-4d6e-ae84-a61e00a3e3b6"
//}),
contentType: "application/json;charset-utf-8",
success: function (data) {
console.log(data);
}
}); var paramObj = { Id: "382accff-57b2-4d6e-ae84-a61e00a3e3b8",Category:}; $.ajax({
type: "post",
url: "/goods/AddGood",
dataType: "json",
//data: null,
data: JSON.stringify({
paramObj: JSON.stringify(paramObj)
}),
contentType: "application/json;charset-utf-8",
success: function (data) {
console.log(data);
}
});

4.WebBrowser

相对来说,这是一种最简单的获取方式。拖WebBrowser控件进去,然后匹配下面这段代码:
WebBrowser web = new WebBrowser();
web.Navigate("http://www.163.com");
web.DocumentCompleted += new WebBrowserDocumentCompletedEventHandler(web_DocumentCompleted);
void web_DocumentCompleted(object sender, WebBrowserDocumentCompletedEventArgs e)
{
WebBrowser web = (WebBrowser)sender;
HtmlElementCollection ElementCollection = web.Document.GetElementsByTagName("Table");
foreach (HtmlElement item in ElementCollection)
{
File.AppendAllText("Kaijiang_xj.txt", item.InnerText);
}
}

拓展(自己总结):WebRequset和WebResponse

#region Post&&Get独立的方式
#region Post /// <summary>
/// Post
/// </summary>
/// <param name="url">url地址</param>
/// <param name="soapbody">提交的数据(XML字符串)</param>
/// <param name="HeadersInfo">头信息</param>
/// <param name="ContentType">格式</param>
/// <param name="userName">账号</param>
/// <param name="passWord">密码</param>
/// <returns></returns>
public static String PostHTTP(string url, string soapbody, WebHeaderCollection HeadersInfo, string userName, string passWord)
{
string Result = string.Empty;
try
{
HttpWebRequest webRequst = (HttpWebRequest)HttpWebRequest.Create(url);//创建URL
webRequst.Method = "POST";//Post方式
webRequst.Headers.Add(HeadersInfo);
SetHeaderValue(webRequst.Headers, "Connection", "keep-alive");
SetHeaderValue(webRequst.Headers, "Content-Length", soapbody.Length.ToString());
SetHeaderValue(webRequst.Headers, "Accept", "*/*");
SetHeaderValue(webRequst.Headers, "Cache-Control", "no-cache");
SetHeaderValue(webRequst.Headers, "ProtocolVersion", "1.1");
SetHeaderValue(webRequst.Headers, "User-Agent", "Provider User-Agent name");
SetHeaderValue(webRequst.Headers, "Origin", "xxxxxx");
SetHeaderValue(webRequst.Headers, "Authorization", "Basic " + Convert.ToBase64String(new ASCIIEncoding().GetBytes(userName + ":" + passWord))); //这个在Post的时候,一定要加上,如果服务器返回错误,他还会继续再去请求,不会使用之前的错误数据,做返回数据
webRequst.ServicePoint.Expect100Continue = false; //指定安全协议为3.0
ServicePointManager.SecurityProtocol = SecurityProtocolType.Ssl3; using (Stream s = webRequst.GetRequestStream())
{
using (StreamWriter sw = new StreamWriter(s))
{
sw.Write(soapbody);
}
} HttpWebResponse webResponse = (HttpWebResponse)webRequst.GetResponse();
using (Stream s = webResponse.GetResponseStream())
{
using (StreamReader sr = new StreamReader(s))
{
Result = sr.ReadToEnd();
}
}
}
catch (WebException ex)
{
var states = ((HttpWebResponse)ex.Response).StatusCode;
var descripition = ((HttpWebResponse)ex.Response).StatusDescription;
Result = "WebException =" + ex.Message + "\n WebStatus=" + states + "\n descripition=" + descripition; }
catch (Exception ex)
{
Result = ex.Message;
} return Result;
} #endregion #region Get /// <summary>
/// Get方式
/// </summary>
/// <returns></returns>
public static String GetHTTP(string url, HttpContext httpContext)
{
string queryString = "?";
string Result = string.Empty;
try
{ foreach (string key in httpContext.Request.QueryString.AllKeys)
{
queryString += key + "=" + httpContext.Request.QueryString[key] + "&";
} queryString = queryString.Substring(, queryString.Length - ); HttpWebRequest httpWebRequest = (HttpWebRequest)WebRequest.Create(url + queryString); httpWebRequest.ContentType = "application/json";
httpWebRequest.Method = "GET";
httpWebRequest.Timeout = ; //byte[] btBodys = Encoding.UTF8.GetBytes(body);
//httpWebRequest.ContentLength = btBodys.Length;
//httpWebRequest.GetRequestStream().Write(btBodys, 0, btBodys.Length); HttpWebResponse httpWebResponse = (HttpWebResponse)httpWebRequest.GetResponse();
StreamReader streamReader = new StreamReader(httpWebResponse.GetResponseStream());
string responseContent = streamReader.ReadToEnd(); httpWebResponse.Close();
streamReader.Close(); Result = responseContent;
}
catch (WebException ex)
{
var states = ((HttpWebResponse)ex.Response).StatusCode;
var descripition = ((HttpWebResponse)ex.Response).StatusDescription;
Result = "WebException =" + ex.Message + "\n WebStatus=" + states + "\n descripition=" + descripition;
}
catch (Exception ex)
{
Result = ex.Message;
}
return Result;
} #endregion
#endregion #region Post&&Get公用方式
public static BookingResponse CreateAuth(string url, string HttpMethod, WebHeaderCollection HeadersInfo, string userName, string passWord, String requestBody = "")
{
BookingResponse Result = new BookingResponse() { Status = false, Message = "get method error" };//初始化结果集
string responseBody = string.Empty;//定义响应体
ServicePointManager.SecurityProtocol = (SecurityProtocolType) | (SecurityProtocolType) | (SecurityProtocolType); try
{
HttpWebRequest webRequst = (HttpWebRequest)HttpWebRequest.Create(url);//创建URL
webRequst.Method = HttpMethod;//HTTP请求方式 //webRequst.Headers.Add(HeadersInfo);//加入之前请求信息 webRequst.ContentType = HeadersInfo["ContentType"];
webRequst.Headers["Accept-Encoding"] = HeadersInfo["Accept-Encoding"];
webRequst.AutomaticDecompression = DecompressionMethods.GZip; //这两句用来解决显示乱码 SetHeaderValue(webRequst.Headers, "Connection", "keep-alive");
SetHeaderValue(webRequst.Headers, "Content-Length", requestBody.Length.ToString());
SetHeaderValue(webRequst.Headers, "Cache-Control", "no-cache");
SetHeaderValue(webRequst.Headers, "Origin", "xxxxxx");
SetHeaderValue(webRequst.Headers, "User-Agent", "Provider User-Agent name");
//SetHeaderValue(webRequst.Headers, "Content-Type", "text/xml;charset=UTF-8");
SetHeaderValue(webRequst.Headers, "Accept", "*/*");
//SetHeaderValue(webRequst.Headers, "Accept-Encoding", "gzip,deflate");
SetHeaderValue(webRequst.Headers, "Authorization", "Basic " + Convert.ToBase64String(new ASCIIEncoding().GetBytes(userName + ":" + passWord)));
SetHeaderValue(webRequst.Headers, "ProtocolVersion", "1.1"); HttpMethod = HttpMethod.Trim().ToUpper(); switch (HttpMethod)
{
case "GET":
responseBody = HttpGet(url);
break;
default:
webRequst.ServicePoint.Expect100Continue = false; //这个在Post的时候,一定要加上,如果服务器返回错误,他还会继续再去请求,不会使用之前的错误数据,做返回数据
responseBody = HttpPost(webRequst, requestBody);
break;
}
if (!string.IsNullOrWhiteSpace(responseBody) && responseBody.Length > && responseBody.Substring(, ) != "error")
{
Result.Status = true;
Result.Message = "success";
Result.Data = responseBody; LogHelper.WriteLog(responseBody);//记录正常日志
}
else
{
if (string.IsNullOrWhiteSpace(responseBody))
Result.Message = "error";
else
Result.Message = responseBody;
}
}
catch (Exception ex)
{
LogHelper.WriteLog("错误日志记录:", ex);
Result.Message = ex.Message.ToString() + responseBody;
Result.Status = false;
}
return Result;
} /// <summary>
/// 发送GET请求
/// </summary>
/// <param name="url"></param>
/// <param name="refer"></param>
/// <returns></returns>
public static string HttpGet(string url)
{
return null;
} /// <summary>
/// 发送Post请求
/// </summary>
/// <param name="webRequst"></param>
/// <returns></returns>
public static string HttpPost(HttpWebRequest webRequst, string requsetbody)
{
string Result = string.Empty;
try
{
using (Stream s = webRequst.GetRequestStream())
{
using (StreamWriter sw = new StreamWriter(s))
{
sw.Write(requsetbody);
}
} HttpWebResponse webResponse = (HttpWebResponse)webRequst.GetResponse();
using (Stream s = webResponse.GetResponseStream())
{
using (StreamReader sr = new StreamReader(s))
{
Result = sr.ReadToEnd();
}
}
}
catch (WebException ex)
{
var states = ((HttpWebResponse)ex.Response).StatusCode;
var descripition = ((HttpWebResponse)ex.Response).StatusDescription;
Result = "WebException =" + ex.Message + "\n WebStatus=" + states + "\n descripition=" + descripition; }
catch (Exception ex)
{
Result = ex.Message;
}
return Result;
} //Uri url = new Uri(urlstring);//初始化uri资源
//string requestMethod = HttpMethod;//设置请求方式 Post Get
//string timezone = DateTime.UtcNow.ToString("s") + "Z";//获取计算机上的时间
//string path = url.PathAndQuery.Split('?')[0];//获取URI的绝对路径
//string query = url.PathAndQuery.Split('?').Count() > 1 ? url.PathAndQuery.Split('?')[1] : "";//获取查询条件
#endregion #region 请求报头设置 /// <summary>
/// 请求报头设置
/// </summary>
/// <param name="header"></param>
/// <param name="name"></param>
/// <param name="value"></param>
public static void SetHeaderValue(WebHeaderCollection header, string name, string value)
{
var property = typeof(WebHeaderCollection).GetProperty("InnerCollection", BindingFlags.Instance | BindingFlags.NonPublic);
if (property != null)
{
var collection = property.GetValue(header, null) as NameValueCollection;
collection[name] = value;
}
} #endregion

借鉴链接:https://blog.csdn.net/jintougao/article/details/12948633#

          https://www.cnblogs.com/lgx5/p/9446763.html

       https://blog.csdn.net/zhailihua/article/details/75048992

通过HTTP请求WEBAPI的方式的更多相关文章

  1. 请求WebApi的几种方式

    目前所了解的请求WebAPI的方式有通过后台访问api 和通过js 直接访问api接口 首先介绍下通过后台访问api的方法,可以使用HttpClient的方式也可以使用WebRequest的方式 1. ...

  2. 十一、Struts2封装请求参数的方式

    十一.Struts2封装请求参数的方式 方式一.Action 本身作为model对象,通过成员setter封装(一个名字为params的拦截器干的) 注意:表单中的名称要和动作类中的名称一致(这是必须 ...

  3. IOS 请求服务器的方式

    IOS 中请求服务器的方式主要有Get 和Post . Get :[1]向服务器发索取数据的一种请求; [2]获取信息,而不是修改信息,类似数据库查询功能一样,数据不会被修改;请求的参数会跟在url后 ...

  4. #WEB安全基础 : HTTP协议 | 0x9 GET和POST请求以及请求URI的方式

    请求URI的方式 1.URI为完整的请求URI GET http://hackr.jp/index.htm HTTP/1.1 2.在首部字段Host中写明域名或IP地址 GET/index.htm H ...

  5. JQuery的Ajax跨域请求的解决方式

            今天在项目中须要做远程数据载入并渲染页面,直到开发阶段才意识到ajax跨域请求的问题,隐约记得Jquery有提过一个ajax跨域请求的解决方式,于是即刻翻出Jquery的API出来研究 ...

  6. 使用Fiddler工具发送post请求(带有json数据)以及get请求(Header方式传参)

    Fiddler工具是一个http协议调试代理工具,它可以帮助程序员测试或调试程序,辅助web开发. Fiddler工具可以发送向服务端发送特定的HTTP请求以及接受服务器回应的请求和数据,是web调试 ...

  7. Servlet请求参数的方式

    今天整理了以下几种常用的Servlet请求参数的方式,下面简单地介绍 1)getParameter(String key)返回一个字符串,获得name和key 一样的表单控件的数据,如果有重复的nam ...

  8. 入门servlet:request获取请求参数通用方式

    一.获取请求参数通用方式 1. String getParameter(String name):根据参数名称获取参数值 例:username=flypig&password=2343 2. ...

  9. google插件跨域含用户请求WebApi解决的方案

    问题描述: google插件跨域请求WebApi相关解决方案 1.ajax解决含登录用户信息 $.ajax({ url: url, type: "POST", timeout: 6 ...

随机推荐

  1. node JS 微信开发

    JS-SDK 要点 微信测试号; 扫码登录;无需认证(只是名称统一为微信测试号)http://mp.weixin.qq.com/debug/cgi-bin/sandbox?t=sandbox/logi ...

  2. caffe2 安装与介绍

    http://blog.csdn.net/yan_joy/article/details/70241319 标签: 深度学习 2017-04-19 15:31 5970人阅读 评论(0) 收藏 举报 ...

  3. SpringMVC学习(一):搭建SpringMVC-注解-非注解

    文章参考:http://www.cnblogs.com/Sinte-Beuve/p/5730553.html 一.环境搭建: 目录结构: 引用的JAR包: 如果是Maven搭建的话pom.xml配置依 ...

  4. php pack()函数详解与示例

    pack和unpack在一般的程序中还真的不容易见到,但是如果你用过很久以前的php生成excel你就会知道了.他的excel的头就是pack出来的最近在尝试与C交互的时候又用上了这玩意,所以不得不再 ...

  5. 软考考点---CPU

    软考考点---CPU         中央处理器(CPU,Central Processing Unit)是一块超大规模的集成电路,是一台计算机的运算核心(Core)和控制核心( Control Un ...

  6. properties配置文件读取

    1.配置文件test.properties: test_123=admin 注:value 可用单引号,双引号,不用引号修饰 2.工具类PropertiesUtil: package com..... ...

  7. IOS-RSA加解密分享

    本文转载至 http://www.cocoachina.com/bbs/read.php?tid=235527     搜索了很多资料,没找到合适的RSA方法,很多人在问这问题,解决了的同志也不分享, ...

  8. 九度OJ 1139:最大子矩阵 (矩阵运算、缓存)

    时间限制:1 秒 内存限制:32 兆 特殊判题:否 提交:1014 解决:376 题目描述: 已知矩阵的大小定义为矩阵中所有元素的和.给定一个矩阵,你的任务是找到最大的非空(大小至少是1 * 1)子矩 ...

  9. Interface AutoCloseable

    https://docs.oracle.com/javase/tutorial/essential/exceptions/try.html https://docs.oracle.com/javase ...

  10. Snow White,摘自iOS应用Snow White and more stories

    Once upon a time, there was a land. 从前,有个国度. It was ruled by an evil queen. 它被一位邪恶的女王统治. Every day t ...