Robot Framework 快速入门_英文版

Copyright © Nokia Siemens Networks 2008

Licensed under the Apache License, Version 2.0

Table of Contents

• Introduction
  • Overview
  • Installation
  • Running this demo
  • Introducing the sample application
• Test cases
  • First test cases
  • Higher-level test cases
  • Data-driven test cases
• Keywords
  • Built-in keywords
  • Library keywords
  • User-defined keywords
• Variables
  • Defining Variables
  • Using variables
• Organizing test cases
  • Test suites
  • Setup and teardown
  • Using tags
• Creating test libraries

Introduction

Overview

Robot Framework is a keyword-driven test automation framework. Test cases live in HTML or TSV (tab-separated values) test files and make use of keywords implemented in test libraries to drive the software under test. Because Robot Framework is flexible and extensible, it is ideally suited to testing complex software with a variety of interfaces: user interfaces, command line, web services, proprietary programming interfaces, etc.

Robot Framework is open source software and installation packages, source code and further documentation is available through http://robotframework.org. This guide is designed to introduce the basic concepts of Robot Framework. For a detailed technical description, please refer to Robot Framework User Guide.

Installation

A precondition for installing Robot Framework is having Python installed. In addition, to use test libraries written in Java, Jython must be installed. To use this Quick Start Guide, Python is enough.

There are three different ways to install Robot Framework, of which the most suitable can be chosen. Detailed installation instructions can be found from the project web pages.

1. There is a binary installer for Windows platform. It is enough to double-click the installer and follow instructions.
2. On every platform, Robot Framework can be installed from source. To install from source, extract the source distribution and run command python setup.py install.
3. If Python package management system Easy Install is available, Robot Framework can be installed by issuing command easy_install robotframework. On Windows you also need to run robot_postinstall.py script manually.

After the framework is installed, it is useful to include the directory containing start-up scripts in PATH environment variable. On UNIX-like systems this should actually happen automatically, but on Windows this must be done from Control Panel > System > Advanced > Environment Variables by adding e.g. C:\Python25\Scripts to PATH.

Successful installation can be verified with command pybot --version which should output something like:

$ pybot --version
Robot Framework 2.0.3 (Python 2.5.2 on linux2)

Running this demo

This Quick Start Guide also acts as an executable demo. To run it, open a command shell, cd to the directory where this file is, and type the following command at the command line:

pybot quickstart.html

The tests in this file will execute and generate the following reports:

report.html

the test results summary

log.html

the test results details

output.xml

the test results in a portable XML format for integration with other tools

Open report.html (the link works only after this guide has been executed) in your browser, then click on the links to explore the results. The report.html file links to the log.html file.

There are also a number of command line options that can be used to control the test execution and generated outputs. Complete list can be viewed by issuing pybot --help. For example the following command changes the name of the log file and the name of the top level test suite:

pybot --log mylog.html --name My_Fine_Tests quickstart.html

Note

Executing this quickstart guide with jybot startup script does not work with Jython 2.2. Executing with Jython 2.5 requires setting correct executable by using -Dsys.executable option.

Introducing the sample application

The sample application for this guide is a variation on a classic login example: it is a command-line based authentication server written in Python. At the moment, the sample application allows a user to do three things:

• Create an account with a valid password.
• Log in with a valid user name and password.
• Change the password of an existing account.

The application itself is in the sut directory and can be executed with a command python sut/login.py. Attempting to log in with a non-existent user account or with an invalid password results in the same error message:

> python sut/login.py login nobody P4ssw0rd
Access Denied

After creating a user account with valid password login succeeds:

> python sut/login.py create fred P4ssw0rd
SUCCESS

> python sut/login.py login fred P4ssw0rd
Logged In

There are two requirements that a password must fulfill to be valid: it must be between 7-12 characters long, and it must contain lower and upper case letters and numbers, but it must not contain special characters. Trying to create a user with invalid password fails:

> python sut/login.py create fred short
Creating user failed: Password must be 7-12 characters long

> python sut/login.py create fred invalid
Creating user failed: Password must be a combination of lowercase and
uppercase letters and numbers

Changing password with invalid credentials results in the same error message as logging in with invalid credentials. The validity of new password is verified and if not valid, an error message is given:

> python sut/login.py change-password fred wrong NewP4ss
Changing password failed: Access Denied

> python sut/login.py change-password fred P4ssw0rd short
Changing password failed: Password must be 7-12 characters long

> python sut/login.py change-password fred P4ssw0rd NewP4ss
SUCCESS

The application uses a simple database file to keep track on user statuses. The file is located in operating system dependent temporary directory.

Test cases

First test cases

Robot Framework test cases are created using a simple tabular syntax. For example the following table has two tests:

• User can create an account and log in
• User cannot log in with bad password

Test Case	Action	Argument	Argument
User can create an account and log in	Create Valid User	fred	P4ssw0rd
	Attempt to Login with Credentials	fred	P4ssw0rd
	Status Should Be	Logged In
User cannot log in with bad password	Create Valid User	betty	P4ssw0rd
	Attempt to Login with Credentials	betty	wrong
	Status Should Be	Access Denied

Notice that these tests read almost like manual tests written in English rather than like automated test cases. Robot Framework uses the keyword-driven approach that supports writing tests that capture the essence of the actions and expectations in natural language. Test cases are constructed from keywords (normally in the second column) and their possible arguments.

Higher-level test cases

Test cases can also be created using only high-level keywords that take no arguments. This style allows using totally free text which is suitable for communication even with non-technical customers or other stakeholders. Robot Framework does not enforce any particular style for writing test cases, and it is possible to use for example given-when-then format popularized by behavior-driven development (BDD) like in the example below.

Test Case	Steps
User can change password	Given a user has a valid account
	when she changes her password
	then she can log in with the new password
	and she cannot use the old password anymore

This kind of use-case or user-story-like test cases are ideally suited for acceptance test-driven development (ATDD). In ATDD acceptance tests are written before implementing actual product features and they act also as requirements.

Data-driven test cases

Quite often several test cases are otherwise similar but they have slightly different input or output data. In these situations data-driven test cases, like six tests below, allow varying the test data without duplicating the workflow.

Test Case	Action	Password	Expected error message
Too short password	Creating user with invalid password should fail	abCD5	${PWD INVALID LENGTH}
Too long password	Creating user with invalid password should fail	abCD567890123	${PWD INVALID LENGTH}
Password without lowercase letters	Creating user with invalid password should fail	123DEFG	${PWD INVALID CONTENT}
Password without capital letters	Creating user with invalid password should fail	abcd56789	${PWD INVALID CONTENT}
Password without numbers	Creating user with invalid password should fail	AbCdEfGh	${PWD INVALID CONTENT}
Password with special characters	Creating user with invalid password should fail	abCD56+	${PWD INVALID CONTENT}

In these tests there is only one keyword per test case, and it is responsible for trying to create a user with the provided password and checking that creation fails with an expected error message. Notice that the error messages are specified using variables.

Keywords

Test cases are created from keywords that can come from three sources: built-in keywords are always available, library keywords come from imported test libraries, and so called user keywords can be created using the same tabular syntax that is used for creating test cases.

Built-in keywords

Some generally useful keywords such as Get Time and Should Be Equal are always available. Technically these keywords come from a test library called BuiltIn and you can see its documentation for a complete list of available keywords.

Library keywords

All lowest level keywords are defined in test libraries which are implemented using standard programming languages. Robot Framework comes with a handful of libraries including an OperatingSystem library to support common operating system functions, and a Screenshot library for taking screenshots. In addition to these standard libraries, there are other libraries distributed in separate open source projects, such as SeleniumLibrary for Web testing. It is also easy to implement your own libraries when there is no suitable library available.

To be able to use keywords provided by a test library, it must be taken into use. Tests in this file need keywords from the standard OperatingSystem library (e.g. Remove File) as well as from a custom made LoginLibrary (e.g. Attempt to login with credentials). Both of these libraries are imported in so called setting table below.

Setting	Value
Library	OperatingSystem
Library	testlibs/LoginLibrary.py

User-defined keywords

One of the most powerful features of Robot Framework is the ability to easily create new higher-level keywords from other keywords. The syntax for creating these so called user-defined keywords, or user keywords for short, is similar to the syntax that is used for creating test cases. All the higher-level keywords needed in previous test cases are created in the keyword table below.

Keyword	Action	Argument	Argument
Clear login database	Remove file	${DATABASE FILE}
Create valid user	[Arguments]	${username}	${password}
	Create user	${username}	${password}
	Status should be	SUCCESS
Creating user with invalid password should fail	[Arguments]	${password}	${error}
	Create user	example	${password}
	Status should be	Creating user failed: ${error}
Login	[Arguments]	${username}	${password}
	Attempt to login with credentials	${username}	${password}
	Status should be	Logged In
# Used by BDD test cases (this is a comment)
Given a user has a valid account	Create valid user	${USERNAME}	${PASSWORD}
When she changes her password	Change password	${USERNAME}	${PASSWORD}
	...	${NEW PASSWORD}
	Status should be	SUCCESS
Then she can log in with the new password	Login	${USERNAME}	${NEW PASSWORD}
And she cannot use the old password anymore	Attempt to login with credentials	${USERNAME}	${PASSWORD}
	Status should be	Access Denied

User-defined keywords can include actions defined by other user-defined keywords, built-in keywords, or library keywords. As you can see from this example, user-defined keywords can take parameters. They can also return values and even contain FOR loops. For now, the important thing to know is that user-defined keywords enable test creators to create reusable steps for common action sequences. User-defined keywords can also help the test author keep the tests as readable as possible and use appropriate abstraction levels in different situations.

Variables

Defining Variables

Variables are an integral part of Robot Framework. Usually any data used in tests that is subject to change is best defined as variables. Syntax for variable definition is quite simple, as seen in this table:

Variable	Value
${USERNAME}	janedoe
${PASSWORD}	J4n3D0e
${NEW PASSWORD}	e0D3n4J
${DATABASE FILE}	${TEMPDIR}${/}robotframework-quickstart-db.txt
${PWD INVALID LENGTH}	Password must be 7-12 characters long
${PWD INVALID CONTENT}	Password must be a combination of lowercase and uppercase letters and numbers

Variables can also be given from the command line which is useful if the tests need to be executed in different environments. For example this demo can be executed like:

pybot --variable USERNAME:johndoe --variable PASSWORD:J0hnD0e quickstart.html

In addition to user defined variables, there are some built-in variables that are always available. These variables include ${TEMPDIR} and ${/} which are used in the above table.

Using variables

Variables can be used in most places in the test data. They are most commonly used as arguments to keywords like the following test case demonstrates. Return values from keywords can also be assigned to variables and used later. For example following Database Should Contain user keyword sets database content to ${database} variable and then verifies the content using built-in keyword Should Contain. Both library and user defined keywords can return values.

Test Case	Action	Argument	Argument	Argument
User status is stored in database	[Tags]	variables	database
	Create Valid User	${USERNAME}	${PASSWORD}
	Database Should Contain	${USERNAME}	${PASSWORD}	Inactive
	Login	${USERNAME}	${PASSWORD}
	Database Should Contain	${USERNAME}	${PASSWORD}	Active

Keyword	Action	Argument	Argument	Argument
Database Should Contain	[Arguments]	${username}	${password}	${status}
	${database} =	Get File	${DATABASE FILE}
	Should Contain	${database}	${username}\t${password}\t${status}

Organizing test cases

Test suites

Collections of test cases are called test suites in Robot Framework. Every input file which contains test cases forms a test suite. When running this demo, you see test suite Quickstart in the console output. This name is got from the file name and it is also visible in the report and log.

It is possible to organize test cases hierarchically by placing test case files into directories and these directories into other directories. All these directories automatically create higher level test suites that get their names from directory names. Since test suites are just files and directories, they are trivially placed into any version control system.

You can test running a directory as a test suite by running following command in the directory where this guide is located:

pybot .

Setup and teardown

If you want a set of actions to occur before and after each test executes, use the Test Setup and Test Teardown settings like so:

Setting	Value
Test Setup	Clear Login Database
Test Teardown

Similarly you can use the Suite Setup and Suite Teardown settings to specify actions to be executed before and after an entire test suite executes.

Using tags

Robot Framework allows setting tags for test cases to give them free metadata. Tags can be set for all test cases in a file with Default Tags or Force Tags settings like in the table below. It is also possible to define tags for single test case like in earlier User status is stored in database test.

Setting	Value	Value
Force Tags	quickstart
Default Tags	example	smoke

When you look at a report after test execution, you can see that tests have specified tags associated with them and there are also statistics generated based on tags. Tags can also be used for many other purposes, one of the most important being the possibility to select what tests to execute. You can try for example following commands:

pybot --include smoke quickstart.html
pybot --exclude database quickstart.html

Creating test libraries

Robot Framework offers a simple API for creating test libraries, both with Python and Java. The user guide contains detailed description with examples.

Below is the source code of LoginLibrary test library used in this guide. You can see, for example, how the keyword Create User is mapped to actual implementation of method create_user.

import os
import sys

class LoginLibrary:

    def __init__(self):
        self._sut_path = os.path.join(os.path.dirname(__file__),
                                      '..', 'sut', 'login.py')
        self._status = ''

    def create_user(self, username, password):
        self._run_command('create', username, password)

    def change_password(self, username, old_pwd, new_pwd):
        self._run_command('change-password', username, old_pwd, new_pwd)

    def attempt_to_login_with_credentials(self, username, password):
        self._run_command('login', username, password)

    def status_should_be(self, expected_status):
        if expected_status != self._status:
            raise AssertionError("Expected status to be '%s' but was '%s'"
                                  % (expected_status, self._status))

    def _run_command(self, command, *args):
        command = '"%s" %s%s' % (self._sut_path, command, ' '.join(args))
        process = os.popen(command)
        self._status = process.read().strip()
        process.close()