Network - curl和Wget

curl - transfer a URL

curl

SYNOPSIS

curl [options] [URL...]

DESCRIPTION

curl is a tool to transfer data from or to a server, using one of the supported protocols (DICT, FILE, FTP, FTPS, GOPHER, HTTP, HTTPS, IMAP, IMAPS, LDAP, LDAPS, POP3, POP3S, RTMP, RTSP,SCP, SFTP, SMTP, SMTPS, TELNET and TFTP).

The command is designed to work without user interaction.

curl offers a busload of useful tricks like proxy support, user authentication, FTP upload, HTTP post, SSL connections, cookies, file transfer resume, Metalink, and more.

curl is powered by libcurl for all transfer-related features. See libcurl(3) for details.

Wget - The non-interactive network downloader.

wget

SYNOPSIS

wget [option]... [URL]...

DESCRIPTION

GNU Wget is a free utility for non-interactive download of files from the Web.

It supports HTTP, HTTPS, and FTP protocols, as well as retrieval through HTTP proxies.

Wget is non-interactive, meaning that it can work in the background, while the user is not logged on.

This allows you to start a retrieval and disconnect from the system, letting Wget finish the work.

By contrast, most of the Web browsers require constant user's presence, which can be a great hindrance when transferring a lot of data.

Wget can follow links in HTML, XHTML, and CSS pages, to create local versions of remote web sites, fully recreating the directory structure of the original site.

This is sometimes referred to as "recursive downloading." While doing that, Wget respects the Robot Exclusion Standard (/robots.txt).

Wget can be instructed to convert the links in downloaded files to point at the local files, for offline viewing.

Wget has been designed for robustness over slow or unstable network connections; if a download fails due to a network problem, it will keep retrying until the whole file has been retrieved.

If the server supports regetting, it will instruct the server to continue the download from where it left off.

区别

	curl	wget
简述	主要作为调试工具	主要作为下载工具
特点	由libcurl库支持，可模拟web请求，测试API，支持协议众多，支持下载	命令行命令，使用简洁，支持递归和断点
缺点	拼接复杂请求麻烦；下载需要额外参数	不支持并发、p2p、磁力链接等

代理设置

临时方式

• 使用命令时指定代理
  
  curl -x "http://10.144.1.10:8080" www.baidu.com

wget -c "http_proxy=10.144.1.10:8080" http://mirrors.sohu.com/python/2.7.12/Python-2.7.12.tgz

• 更改当前终端的环境变量
  
  一般的linux程序都是使用http_proxy和ftp_proxy这两个环境变量来获得代理设置的。
  
  声明环境变量export http_proxy=proxy_addr:port就可以在这个shell界面直接使用命令了。
  
  例如：export http_proxy=10.144.1.10:8080

永久方式

在配置文件/etc/wgetrc中添加如下内容:

# You can set the default proxies for Wget to use for http and ftp.
# They will override the value in the environment.
http_proxy = http://10.144.1.10:8080/
https_proxy = http://10.144.1.10:8080/
#ftp_proxy = http://proxy.yoyodyne.com:18023/

# If you do not want to use proxy at all, set this to off.
use_proxy = yes