curl模拟多线程

1.curl发送请求的请求的基本思路

1. 打开会话
2. 设置会话参数
3. 发送请求
4. 关闭句柄

2.curl单进程发送示例

<?php
//请求链接
$arr = [
    'https://www.yahoo.com/',
    'http://www.jtthink.com/',
    'https://www.hao123.com/',
    'http://www.cnblogs.com/loveyouyou616/',
    'http://www.qq.com/',
    'http://www.sina.com.cn/',
    'http://www.163.com/',
    'https://www.yahoo.com/',
    'http://www.jtthink.com/',
    'https://www.hao123.com/',
    'http://www.cnblogs.com/loveyouyou616/',
    'http://www.qq.com/',
    'http://www.sina.com.cn/',
    'http://www.163.com/',
    'https://www.yahoo.com/',
    'http://www.jtthink.com/',
    'https://www.hao123.com/',
    'http://www.cnblogs.com/loveyouyou616/',
    'http://www.qq.com/',
    'http://www.sina.com.cn/',
    'http://www.163.com/'
];
$headers = ['User-Agent:Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/32.0.1700.107 Safari/537.36',];
foreach ($arr as $key => $url) {
    //打开curl会话
    $ch = curl_init($url);
    //设置会话参数
    //设置curl传输选项
    curl_setopt($ch,CURLOPT_RETURNTRANSFER,1);
    curl_setopt($ch, CURLOPT_HTTPHEADER,$headers);
    curl_setopt($ch, CURLOPT_HEADER, 0);
    curl_setopt($ch, CURLOPT_TIMEOUT, 20);
    //关闭https请求验证
    if (strpos($url,'https')){
        curl_setopt ( $ch, CURLOPT_SSL_VERIFYPEER, false );
        curl_setopt ( $ch, CURLOPT_SSL_VERIFYHOST, 2 );
    }
    //执行curl会话
    $res = curl_exec($ch);
    //关闭curl会话并释放资源
    curl_close($ch);
}

3.curl模拟多线程发送实例

<?php
//脚本开始的毫秒时刻
$start = microtime(true);
//链接数组
$arr = [
  'https://www.yahoo.com/',
  'http://www.jtthink.com/',
  'https://www.hao123.com/',
  'http://www.cnblogs.com/loveyouyou616/',
  'http://www.qq.com/',
  'http://www.sina.com.cn/',
  'http://www.163.com/',
  'https://www.yahoo.com/',
  'http://www.jtthink.com/',
  'https://www.hao123.com/',
  'http://www.cnblogs.com/loveyouyou616/',
  'http://www.qq.com/',
  'http://www.sina.com.cn/',
  'http://www.163.com/',
  'https://www.yahoo.com/',
  'http://www.jtthink.com/',
  'https://www.hao123.com/',
  'http://www.cnblogs.com/loveyouyou616/',
  'http://www.qq.com/',
  'http://www.sina.com.cn/',
  'http://www.163.com/'
];
$headers = ['User-Agent:Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/32.0.1700.107 Safari/537.36',];
//打开一个curl批处理句柄
$mh = curl_multi_init();
foreach ($arr as $key => $url) {
  //初始化cURL会话
  $ch[$key] = curl_init($url);
  //设置curl传输选项
  curl_setopt($ch[$key],CURLOPT_RETURNTRANSFER,1);
  curl_setopt($ch[$key], CURLOPT_HTTPHEADER,$headers);
  curl_setopt($ch[$key], CURLOPT_HEADER, 0);
  curl_setopt($ch[$key], CURLOPT_TIMEOUT, 20);
  //关闭https请求验证
  if (strpos($url,'https')){
  curl_setopt ( $ch[$key], CURLOPT_SSL_VERIFYPEER, false );
  curl_setopt ( $ch[$key], CURLOPT_SSL_VERIFYHOST, 2 );
 }  //向批处理句柄中添加单独的curl句柄
  curl_multi_add_handle($mh, $ch[$key]);
}
$running = null;
//执行批处理句柄
do {
  curl_multi_exec($mh, $running);
} while($running > 0);
//获取内容
foreach ($arr as $key => $url) {
  //返回获取的输出文本流
  $res[$key] = curl_multi_getcontent($ch[$key]);
  //关闭执行完的子句柄
  curl_close($ch[$key]);
}
$end = microtime(true) - $start;
file_put_contents(__DIR__ . '/exec_time.log', $end . PHP_EOL, FILE_APPEND);
echo $end;</pre>

4.说明

• 上面的模型也存在一个问题，返回的时间依赖最长的请求，通俗的说就是结果数据要等所有http请求结束后一起返回，然后逐个处理数据。