Snoopy - the PHP net client v1.2.4

Snoopy是一个php类,用来模拟浏览器的功能,可以获取网页内容,发送表单。
Snoopy的特点:
1、抓取网页的内容 fetch
2、抓取网页的文本内容 (去除HTML标签) fetchtext
3、抓取网页的链接,表单 fetchlinks fetchform
4、支持代理主机
5、支持基本的用户名/密码验证
6、支持设置 user_agent, referer(来路), cookies 和 header content(头文件)
7、支持浏览器重定向,并能控制重定向深度
8、能把网页中的链接扩展成高质量的url(默认)
9、提交数据并且获取返回值
10、支持跟踪HTML框架
11、支持重定向的时候传递cookies
要求php4以上就可以了,由于本身是php一个类,无需扩支持,服务器不支持curl时候的最好选择。

概要方法:

    include "Snoopy.class.php";
$snoopy = new Snoopy; $snoopy->fetchtext("http://www.php.net/");
print $snoopy->results; $snoopy->fetchlinks("http://www.phpbuilder.com/");
print $snoopy->results; $submit_url = "http://lnk.ispi.net/texis/scripts/msearch/netsearch.html"; $submit_vars["q"] = "amiga";
$submit_vars["submit"] = "Search!";
$submit_vars["searchhost"] = "Altavista"; $snoopy->submit($submit_url,$submit_vars);
print $snoopy->results; $snoopy->maxframes=5;
$snoopy->fetch("http://www.ispi.net/");
echo "<PRE>\n";
echo htmlentities($snoopy->results[0]);
echo htmlentities($snoopy->results[1]);
echo htmlentities($snoopy->results[2]);
echo "</PRE>\n"; $snoopy->fetchform("http://www.altavista.com");
print $snoopy->results; 

类方法说明:

    fetch($URI)
----------- This is the method used for fetching the contents of a web page.
$URI is the fully qualified URL of the page to fetch.
The results of the fetch are stored in $this->results.
If you are fetching frames, then $this->results
contains each frame fetched in an array. fetchtext($URI)
--------------- This behaves exactly like fetch() except that it only returns
the text from the page, stripping out html tags and other
irrelevant data. fetchform($URI)
--------------- This behaves exactly like fetch() except that it only returns
the form elements from the page, stripping out html tags and other
irrelevant data. fetchlinks($URI)
---------------- This behaves exactly like fetch() except that it only returns
the links from the page. By default, relative links are
converted to their fully qualified URL form. submit($URI,$formvars)
---------------------- This submits a form to the specified $URI. $formvars is an
array of the form variables to pass. submittext($URI,$formvars)
-------------------------- This behaves exactly like submit() except that it only returns
the text from the page, stripping out html tags and other
irrelevant data. submitlinks($URI)
---------------- This behaves exactly like submit() except that it only returns
the links from the page. By default, relative links are
converted to their fully qualified URL form.

类 VARIABLES: (default value in parenthesis)

    $host            the host to connect to
$port the port to connect to
$proxy_host the proxy host to use, if any
$proxy_port the proxy port to use, if any
$agent the user agent to masqerade as (Snoopy v0.1)
$referer referer information to pass, if any
$cookies cookies to pass if any
$rawheaders other header info to pass, if any
$maxredirs maximum redirects to allow. 0=none allowed. (5)
$offsiteok whether or not to allow redirects off-site. (true)
$expandlinks whether or not to expand links to fully qualified URLs (true)
$user authentication username, if any
$pass authentication password, if any
$accept http accept types (image/gif, image/x-xbitmap, image/jpeg, image/pjpeg, )
$error where errors are sent, if any
$response_code responde code returned from server
$headers headers returned from server
$maxlength max return data length
$read_timeout timeout on read operations (requires PHP 4 Beta 4+)
set to 0 to disallow timeouts
$timed_out true if a read operation timed out (requires PHP 4 Beta 4+)
$maxframes number of frames we will follow
$status http status of fetch
$temp_dir temp directory that the webserver can write to. (/tmp)
$curl_path system path to cURL binary, set to false if none

EXample:

    Example:     fetch a web page and display the return headers and
the contents of the page (html-escaped): include "Snoopy.class.php";
$snoopy = new Snoopy; $snoopy->user = "joe";
$snoopy->pass = "bloe"; if($snoopy->fetch("http://www.slashdot.org/"))
{
echo "response code: ".$snoopy->response_code."<br>\n";
while(list($key,$val) = each($snoopy->headers))
echo $key.": ".$val."<br>\n";
echo "<p>\n"; echo "<PRE>".htmlspecialchars($snoopy->results)."</PRE>\n";
}
else
echo "error fetching document: ".$snoopy->error."\n"; Example: submit a form and print out the result headers
and html-escaped page: include "Snoopy.class.php";
$snoopy = new Snoopy; $submit_url = "http://lnk.ispi.net/texis/scripts/msearch/netsearch.html"; $submit_vars["q"] = "amiga";
$submit_vars["submit"] = "Search!";
$submit_vars["searchhost"] = "Altavista"; if($snoopy->submit($submit_url,$submit_vars))
{
while(list($key,$val) = each($snoopy->headers))
echo $key.": ".$val."<br>\n";
echo "<p>\n"; echo "<PRE>".htmlspecialchars($snoopy->results)."</PRE>\n";
}
else
echo "error fetching document: ".$snoopy->error."\n"; Example: showing functionality of all the variables: include "Snoopy.class.php";
$snoopy = new Snoopy; $snoopy->proxy_host = "my.proxy.host";
$snoopy->proxy_port = "8080"; $snoopy->agent = "(compatible; MSIE 4.01; MSN 2.5; AOL 4.0; Windows 98)";
$snoopy->referer = "http://www.microsnot.com/"; $snoopy->cookies["SessionID"] = 238472834723489l;
$snoopy->cookies["favoriteColor"] = "RED"; $snoopy->rawheaders["Pragma"] = "no-cache"; $snoopy->maxredirs = 2;
$snoopy->offsiteok = false;
$snoopy->expandlinks = false; $snoopy->user = "joe";
$snoopy->pass = "bloe"; if($snoopy->fetchtext("http://www.phpbuilder.com"))
{
while(list($key,$val) = each($snoopy->headers))
echo $key.": ".$val."<br>\n";
echo "<p>\n"; echo "<PRE>".htmlspecialchars($snoopy->results)."</PRE>\n";
}
else
echo "error fetching document: ".$snoopy->error."\n"; Example: fetched framed content and display the results include "Snoopy.class.php";
$snoopy = new Snoopy; $snoopy->maxframes = 5; if($snoopy->fetch("http://www.ispi.net/"))
{
echo "<PRE>".htmlspecialchars($snoopy->results[0])."</PRE>\n";
echo "<PRE>".htmlspecialchars($snoopy->results[1])."</PRE>\n";
echo "<PRE>".htmlspecialchars($snoopy->results[2])."</PRE>\n";
}
else
echo "error fetching document: ".$snoopy->error."\n";

Snoopy.class.php使用手册的更多相关文章

  1. Snoopy.class.php介绍

    Snoopy是一个开源的模拟抓取工具,找到一个不错的介绍网页 记录一下: php开源采集类Snoopy.class.php功能使用介绍与下载地址 Snoopy.class.php使用手册 还有一个介绍 ...

  2. simple_html_dom配合snoopy使用

    https://github.com/samacs/simple_html_dom Snoopy的特点是“大”和“全”,一个fetch什么都采到了,可以作为采集的第一步.接下来就需要用simple_h ...

  3. FREERTOS 手册阅读笔记

    郑重声明,版权所有! 转载需说明. FREERTOS堆栈大小的单位是word,不是byte. 根据处理器架构优化系统的任务优先级不能超过32,If the architecture optimized ...

  4. JS魔法堂:不完全国际化&本地化手册 之 理論篇

    前言  最近加入到新项目组负责前端技术预研和选型,其中涉及到一个熟悉又陌生的需求--国际化&本地化.熟悉的是之前的项目也玩过,陌生的是之前的实现仅仅停留在"有"的阶段而已. ...

  5. 转职成为TypeScript程序员的参考手册

    写在前面 作者并没有任何可以作为背书的履历来证明自己写作这份手册的分量. 其内容大都来自于TypeScript官方资料或者搜索引擎获得,期间掺杂少量作者的私见,并会标明. 大部分内容来自于http:/ ...

  6. Redis学习手册(目录)

    为什么自己当初要选择Redis作为数据存储解决方案中的一员呢?现在能想到的原因主要有三.其一,Redis不仅性能高效,而且完全免费.其二,是基于C/C++开发的服务器,这里应该有一定的感情因素吧.最后 ...

  7. JS魔法堂:不完全国际化&本地化手册 之 实战篇

    前言  最近加入到新项目组负责前端技术预研和选型,其中涉及到一个熟悉又陌生的需求--国际化&本地化.熟悉的是之前的项目也玩过,陌生的是之前的实现仅仅停留在"有"的阶段而已. ...

  8. Windows API 函数列表 附帮助手册

    所有Windows API函数列表,为了方便查询,也为了大家查找,所以整理一下贡献出来了. 帮助手册:700多个Windows API的函数手册 免费下载 API之网络函数 API之消息函数 API之 ...

  9. linux命令在线手册

    下面几个网址有一些 Linux命令的在线手册,而且还是中文的,还可以搜索.非常方便 Linux命令手册 Linux命令大全 Linux中文man在线手册 每日一linux命令

随机推荐

  1. 内部排序总结之----插入类排序(插入和Shell)

    一.直接插入排序 直接插入排序(straight insertion sort)的做法是: 每次从无序表中取出第一个元素,把它插入到有序表的合适位置,使有序表仍然有序. 第一趟比较前两个数,然后把第二 ...

  2. Netfilter 之 连接跟踪相关数据结构

    Netfilter通过连接跟踪来记录和跟踪连接的状态,为状态防火墙和NAT提供基础支持: 钩子点与钩子函数 下图为钩子点和钩子函数的关系图(点击图片查看原图),其中ipv4_conntrack_def ...

  3. 【软件工程】Beta冲刺(3/5)

    链接部分 队名:女生都队 组长博客: 博客链接 作业博客:博客链接 小组内容 恩泽(组长) 过去两天完成了哪些任务 描述 新增数据分析展示等功能API 服务器后端部署,API接口的beta版实现 展示 ...

  4. antd源码分析之——折叠面板(collapse)

    官方文档 https://ant.design/components/collapse-cn/ 目录 一.antd中的collapse 代码目录 1.组件结构图(♦♦♦重要) 2.源码节选:antd/ ...

  5. JDK8记FullGC时候Metaspace内存不会被垃圾回收

    本文链接:https://blog.csdn.net/a15939557197/article/details/90635460背景前段时间有一个这样的需求:第三方调用接口,30分钟内调用120W次: ...

  6. Java注解(Annotation)详解

    转: Java注解(Annotation)详解 幻海流心 2018.05.23 15:20 字数 1775 阅读 380评论 0喜欢 1 Java注解(Annotation)详解 1.Annotati ...

  7. JAVA 基础编程练习题11 【程序 11 求不重复数字】

    11 [程序 11 求不重复数字] 题目:有 1.2.3.4 个数字,能组成多少个互不相同且无重复数字的三位数?都是多少? 程序分析:可填在百位.十位.个位的数字都是 1.2.3.4.组成所有的排列后 ...

  8. Vue 子组件与子组件之间传值

    可以借用公共父元素.子组件A  this.$emit("eventName", data) 触发事件,父组件监听事件,更改父组件 data , 通过Props 传值到子组件B,子组 ...

  9. confilicts

    confilicts 矛盾; 冲突 安装软件提升这个表示安装软件之间的冲突,需要卸载冲突的软件再安装

  10. springcloud随便写点

    eureka  注册注册 ribbon 负载均衡 feign  声名式client hystrix  断路器 zuul  网关,智能路由,过滤 config 集群配置 bus 分布式的节点用轻量的消息 ...