Perl & Python编写CGI

近期偶然玩了一下CGI，收集点资料写篇在这里留档。

如今想做HTTP Cache回归測试了，为了模拟不同的响应头及数据大小。就须要一个CGI按须要传回指定的响应头和内容。这是从老外的測试页面学习到的经验。

CGI事实上就是使用STDIN和环境变量作为输入, STDOUT做为输出。按照Http协议生成相应的数据。

一. 数据输出

数据输出遵循HTTP协议，分为四部分:

状态行 (Status Line):

200 OK

响应头(Response Headers):

Content-Type: text/html

Content-Length: 3072

空白行(代表响应头已经列完, 仅仅能包括回车和换行符):

数据(文本或二进制数据):

<html>xxxxxx</html>

參考: HTTP Protocol

状态行和响应头中每一行都必须以\r\n(回车及换行)结束。使用这个规则，以简单的文本组装就能够完毕响应了。

仅仅要注意二进制数据输出方法就能够了。尝试设定Content-Length时可能没办法设定的准确，至少对于Apache Server是这样，只是不影响最后返回的数据量。

二. 数据输入

CGI的数据输入主要是环境变量，以下这个链接有一串定义,

CGI Environment Variables

Key	Value
DOCUMENT_ROOT	The root directory of your server
HTTP_COOKIE	The visitor's cookie, if one is set
HTTP_HOST	The hostname of the page being attempted
HTTP_REFERER	The URL of the page that called your program
HTTP_USER_AGENT	The browser type of the visitor
HTTPS	"on" if the program is being called through a secure server
PATH	The system path your server is running under
QUERY_STRING	The query string (see GET, below)
REMOTE_ADDR	The IP address of the visitor
REMOTE_HOST	The hostname of the visitor (if your server has reverse-name-lookups on; otherwise this is the IP address again)
REMOTE_PORT	The port the visitor is connected to on the web server
REMOTE_USER	The visitor's username (for .htaccess-protected pages)
REQUEST_METHOD	GET or POST
REQUEST_URI	The interpreted pathname of the requested document or CGI (relative to the document root)
SCRIPT_FILENAME	The full pathname of the current CGI
SCRIPT_NAME	The interpreted pathname of the current CGI (relative to the document root)
SERVER_ADMIN	The email address for your server's webmaster
SERVER_NAME	Your server's fully qualified domain name (e.g. www.cgi101.com)
SERVER_PORT	The port number your server is listening on
SERVER_SOFTWARE	The server software you're using (e.g. Apache 1.3)

当你须要CGI处理POST请求时，CGI就要使用STDIN做为输入来接收数据了。

在Perl里使用以下的代码读取:

use CGI;

my $cgi = CGI->new();

my %params = $cgi->Vars();

而在Python则是:

import cgi

cgi.FieldStorage()

參考: Perl CGI Input Test

三. 开发语言

能够写CGI的开发语言太多，以下附上两个分别使用Perl和Python编写的同样功能的CGI, 正好能够做个对照。

这两个脚本能够接收Query String, 然后返回不同的文件。数据大小，以及缓存相关的头信息，能够区分二进制数据和文本数据。

CGI脚本支持以Query String改动以下响应头:

content type,

cache control,

content length (仅仅返回相应大小的数据),

last modified,

expires

以下是返回一个指定大小图片的样例:

/cgi/cache_factory.pl?type=image&size=32&last-modified=Fri, 02 Apr 2014 02:34:06 GMT&cache-control=private,max-age=60&expires=Fri, 22 Apr 2014 02:34:06 GMT

代码非常easy。能够做个參考。

Perl版本号

#!/usr/bin/perl

use strict;

use warnings;

use CGI;

use constant BUFFER_SIZE     => 4_096;

use constant DATA_DIRECTORY => "/var/www/Cache";

my $cgi = CGI->new();

my %params = $cgi->Vars();

&parserCommonHeaders;

if(exists $params{'type'} && $params{'type'}=="image"){

	&generateImageData;

 }

else{

	&generateTextData;

}

sub parserCommonHeaders{

	if(exists $params{'cache-control'}){

	  print 'Cache-Control:',$params{'cache-control'},"\r\n";

	}

	 if(exists $params{'last-modified'}){

	   print 'Last-Modified:',$params{'last-modified'},"\r\n";

	 }

	 if(exists $params{'expires'}){

	   print 'Expires:',$params{'expires'},"\r\n";

	 }

	 if(exists $params{'etag'}){

	   print 'ETag:ea6186e11526ce1:0',"\r\n";

	 }

}

sub generateImageData{

	my $buffer = "";

	my $targetSize = 100*1024*1024;

	if(exists $params{'size'} && $params{'size'}>0){

		$targetSize = 1024*$params{'size'};

		print "Content-length: $targetSize \r\n";

	}

	my $image = DATA_DIRECTORY .'/images/very_big.jpg';

	my( $type ) = $image =~ /\.(\w+)$/;

	$type eq "jpg" and $type == "jpeg";

	print $cgi->header( -type => "image/$type", -expires => "-1d" );

	binmode STDOUT;

	local *IMAGE;

	open IMAGE, $image or die "Cannot open file $image: $!";

	my $sentSize = 0;

	while ( read( IMAGE, $buffer, BUFFER_SIZE ) ) {

	    print $buffer;

	    $sentSize += BUFFER_SIZE;

	    if($sentSize>=$targetSize){

	    	last;

	    }

	}

	close IMAGE;

}

sub generateTextData{

	my $startHeader = '<html><head><title>HTTP Cache Testing HTML</title></head><body>';

	my $tailPart = '</body></html>';

	if(exists $params{'type'}){

		print "Content-type:$params{'type'}\r\n";

	}

	else{

		print "Content-type:text/html\r\n";

	}

	my $targetTextSize = 100*1024*1024;

	if(exists $params{'size'} && $params{'size'}>0){

		$targetTextSize = 1024*$params{'size'};

		print "Content-length: $targetTextSize \r\n";

	}

	print "\r\n";

	$targetTextSize -= length($startHeader) + length($tailPart);

	print "$startHeader";

	my $filepath = DATA_DIRECTORY .'/files/big_text.txt';

	open(FILE, $filepath) or die $!;

	my @lines = <FILE>;

	close(FILE);

	foreach my $line (@lines) {

    	if( length($line)<=$targetTextSize ){

    		print $line;

    	}

    	else{

    		print substr($line,0,$targetTextSize);

    	}

    	$targetTextSize -= length($line);

    	if($targetTextSize<=0){

    		last;

    	}

	}

	print "$tailPart";

}

Python版本号

#!/usr/bin/python

import os

import cgi

import sys

BUFFER_SIZE = 4096

DATA_DIRECTORY = "/var/www/Cache"

def parserCommonHeaders(formQuery):

  if('cache-control' in formQuery.keys()):

    print 'Cache-Control:',formQuery['cache-control'].value,"\r\n",  

  if('last-modified' in formQuery.keys()):

     print 'Last-Modified:',formQuery['last-modified'].value,"\r\n",  

  if('expires' in formQuery.keys()):

     print 'Expires:',formQuery['expires'].value,"\r\n",

  if('etag' in formQuery.keys()):

     print 'ETag:ea6186e11526ce1:0',"\r\n",

def generateImageData(formQuery):

  targetSize = 100*1024*1024;

  if('size' in formQuery.keys()):

    targetSize = 1024*int(formQuery['size'].value)

    print "Content-length:",targetSize,"\r\n",

  image = DATA_DIRECTORY+'/images/very_big.jpg'

  print "Content-Type:image/jpeg\r\n",

  print 

  sentSize = 0

  f = open(image, 'rb')

  while True:

    data = f.read(4096)

    sentSize = sentSize + BUFFER_SIZE

    sys.stdout.write(data)

    if sentSize>targetSize:

        break

  sys.stdout.flush()

  close(f)

def generateTextData(formQuery):

  startHeader = '<html><head><title>HTTP Cache Testing HTML</title></head><body>'

  tailPart = '</body></html>'

  targetTextSize = 2.3*1024*1024;

  if('size' in formQuery.keys()):

    targetTextSize = 1024*int(formQuery['size'].value)

    print "Content-length:",targetTextSize,"\r\n",

  if('type' in formQuery.keys()):

    print "Content-type: %s\r\n"%(formQuery['type'].value),

  else:

    print "Content-type: text/html\r\n",

  print

  print startHeader

  targetTextSize = targetTextSize - len(startHeader) - len(tailPart)

  filepath = DATA_DIRECTORY + '/files/big_text.txt'

  file = open(filepath)

  lines = file.readlines()

  file.close()

  for line in lines:

      if( len(line) <= targetTextSize ):

        print line

      else:

        print line[0:targetTextSize]

      targetTextSize = targetTextSize - len(line)

      if(targetTextSize<=0):

        break

  print tailPart

if __name__ =="__main__":

  formQuery = cgi.FieldStorage() #os.environ['QUERY_STRING']

  parserCommonHeaders(formQuery)

  if('type' in formQuery.keys() and formQuery['type'].value=="image"):

    generateImageData(formQuery)

  else:

    generateTextData(formQuery)

四. 服务器

服务器端使用Apache Server+WANem, 配合CGI完毕灵活的需求，开通SFTP端口供相关同学编辑。方便共享測试用例。

.-----. .-------.

| CGI | | WANem |

'-----'---'-------'

| Apache Server |

'-----------------'

SFTP & HTTP

.------------------.

| Web Page Editor |

| and Browser |

'------------------'

配有WANem最大的优点就是能够依据需求进网络状态调整, 甚至能够用相似以下的方式在測试用例动态调整。

參考:

Apache配置 (不要忘记给CGI脚本加上可运行权限)

CGI Programming 101 (Perl)

Debugging CGI Programs (Perl)

Python CGI编程

Python CGI Debugging