【solr专题之四】关于VelocityResponseWriter

一、关于Velocity的基本配置

在Solr中，可以以多种方式返回搜索结果，如单纯的文本回复（XML、JSON、CSV等），也可以返回velocity，js等格式。而VelocityResponseWriter就是用于将返回velocity类型文本，以便直接用于结果呈现。

在Solr提供的example，其中的一个RequestHandler--/browse，使用了VelocityResponseWriter。其配置如下：

  <requestHandler name="/browse" class="solr.SearchHandler">
     <lst name="defaults">
       <str name="echoParams">explicit</str>
 
       <!-- VelocityResponseWriter settings -->
       <str name="wt">velocity</str>
       <str name="v.template">browse</str>
       <str name="v.layout">layout</str>
       <str name="title">Solritas_test</str>
 
       <!-- Query settings -->
       <str name="defType">edismax</str>
       <str name="qf">
          text^0.5 features^1.0 name^1.2 sku^1.5 id^10.0 manu^1.1 cat^1.4
          title^10.0 description^5.0 keywords^5.0 author^2.0 resourcename^1.0
       </str>
       <str name="df">text</str>
       <str name="mm">100%</str>
       <str name="q.alt">*:*</str>
       <str name="rows">10</str>
       <str name="fl">*,score</str>
 
       <str name="mlt.qf">
         text^0.5 features^1.0 name^1.2 sku^1.5 id^10.0 manu^1.1 cat^1.4
         title^10.0 description^5.0 keywords^5.0 author^2.0 resourcename^1.0
       </str>
       <str name="mlt.fl">text,features,name,sku,id,manu,cat,title,description,keywords,author,resourcename</str>
       <int name="mlt.count">3</int>
 
       <!-- Faceting defaults -->
       <str name="facet">on</str>
       <str name="facet.field">cat</str>
       <str name="facet.field">manu_exact</str>
       <str name="facet.field">content_type</str>
       <str name="facet.field">author_s</str>
       <str name="facet.query">ipod</str>
       <str name="facet.query">GB</str>
       <str name="facet.mincount">1</str>
       <str name="facet.pivot">cat,inStock</str>
       <str name="facet.range.other">after</str>
       <str name="facet.range">price</str>
       <int name="f.price.facet.range.start">0</int>
       <int name="f.price.facet.range.end">600</int>
       <int name="f.price.facet.range.gap">50</int>
       <str name="facet.range">popularity</str>
       <int name="f.popularity.facet.range.start">0</int>
       <int name="f.popularity.facet.range.end">10</int>
       <int name="f.popularity.facet.range.gap">3</int>
       <str name="facet.range">manufacturedate_dt</str>
       <str name="f.manufacturedate_dt.facet.range.start">NOW/YEAR-10YEARS</str>
       <str name="f.manufacturedate_dt.facet.range.end">NOW</str>
       <str name="f.manufacturedate_dt.facet.range.gap">+1YEAR</str>
       <str name="f.manufacturedate_dt.facet.range.other">before</str>
       <str name="f.manufacturedate_dt.facet.range.other">after</str>
 
       <!-- Highlighting defaults -->
       <str name="hl">on</str>
       <str name="hl.fl">content features title name</str>
       <str name="hl.encoder">html</str>
       <str name="hl.simple.pre"><b></str>
       <str name="hl.simple.post"></b></str>
       <str name="f.title.hl.fragsize">0</str>
       <str name="f.title.hl.alternateField">title</str>
       <str name="f.name.hl.fragsize">0</str>
       <str name="f.name.hl.alternateField">name</str>
       <str name="f.content.hl.snippets">3</str>
       <str name="f.content.hl.fragsize">200</str>
       <str name="f.content.hl.alternateField">content</str>
       <str name="f.content.hl.maxAlternateFieldLength">750</str>
 
       <!-- Spell checking defaults -->
       <str name="spellcheck">on</str>
       <str name="spellcheck.extendedResults">false</str>
       <str name="spellcheck.count">5</str>
       <str name="spellcheck.alternativeTermCount">2</str>
       <str name="spellcheck.maxResultsForSuggest">5</str>
       <str name="spellcheck.collate">true</str>
       <str name="spellcheck.collateExtendedResults">true</str>
       <str name="spellcheck.maxCollationTries">5</str>
       <str name="spellcheck.maxCollations">3</str>
     </lst>
 
     <!-- append spellchecking to our list of components -->
     <arr name="last-components">
       <str>spellcheck</str>
     </arr>
  </requestHandler>

关于velocity这个writer的定义如下：

  <!--
     Custom response writers can be declared as needed...
    -->
    <queryResponseWriter name="velocity" class="solr.VelocityResponseWriter" startup="lazy"/>

处理一个流程的步骤如下：

1、根据请求url查找相关的配置

请求的url为：http://localhost:8983/solr/browse

则在solrConfig.xml中查找  /browse的配置，可以得出上述所示的结果。

2、其中有关velocity的内容如下：

  <requestHandler name="/browse" class="solr.SearchHandler">
     <lst name="defaults">
       <str name="echoParams">explicit</str>
       <!-- VelocityResponseWriter settings -->
       <str name="wt">velocity</str>
       <str name="v.template">browse</str>
       <str name="v.layout">layout</str>
       <str name="title">Solritas_test</str>

从上述定义中开始分别查找显示层的内容(vm文件)与处理类的内容(wt的实现类。)

3、定位显示内容模板

根据v.template属性，定义到文件browse.vm，注意在配置中省略了后缀名vm。

由于存在v.layout属性，因此，此属性的值将作为模板，而v.template中的内容将作为$content的内容。

• v.layout: Template name that wraps main template (v.template). Main template renders to a $content that can be used in layout template.

layout.vm的内容如下：

#**
 *  Overall HTML page layout
 *#
 
<html>
<head>
  #parse("head.vm")
</head>
  <body>
    <div id="admin"><a href="#url_root/#/#core_name">Solr Admin</a></div>
    <div id="header">
      #parse("header.vm")
    </div>
    <div id="tabs">
      #parse("tabs.vm")
    </div>
    <div id="content">
      $content
    </div>
    <div id="footer">
      #parse("footer.vm")
    </div>
  </body>
</html>

4、定位ResponseWriter 

 从第2步的结果知道，使用的velocity，然后查找这个wt的定义，可以得到

    <queryResponseWriter name="velocity" class="solr.VelocityResponseWriter" startup="lazy"/>

即VelocityResponseWriter实现类，其定义如下：

public class VelocityResponseWriter
extends Object
implements QueryResponseWriter

与Solr返回Velocity相关的类只有4个：

• PageTool
• SolrParamResourceLoader
• SolrVelocityResourceLoader
• VelocityResponseWriter

以下是关于VelocityResponseWriter的官方说明：http://wiki.apache.org/solr/VelocityResponseWriter

Introduction

VelocityResponseWriter (aka Solritas) enables
Solr to respond with content generated from Velocity templates. Along with technologies like SolrJS, this makes
Solr itself capable of driving sophisticated search interfaces without the need for an intermediate application server between the browser and Solr.

See SOLR-620 for more
information.

Contents

1. Introduction
2. Instructions to use, Solr 1.4+
   1. Sample Usage
      1. Using the VelocityResponseWriter
         in Solr Core
3. Instructions to use, Solr 4.0+
   1. Sample Usage
4. Options (All Versions)
5. Velocity Context
6. TODO

Instructions to use, Solr 1.4+

These steps will get you up and running for the examples below:

• Download and install Solr 1.4.x
• Fire up Solr: cd example; java -Dsolr.solr.home=../contrib/velocity/src/main/solr/ -jar start.jar
• Index sample docs: cd example/exampledocs; java -jar post.jar *.xml
• Hit the examples below...

Sample Usage

http://localhost:8983/solr/itas

• Renders browse.vm from conf/velocity. Faceted navigation included.

http://localhost:8983/solr/itas?v.template.header=Custom%20Header

• Renders browse.vm, but overrides the header.vm from conf/velocity with the specified value.

http://localhost:8983/solr/itas?debugQuery=true

• Renders browse.vm, adding in explanation views per hit, and a Velocity context dump at the end.

Using the VelocityResponseWriter in Solr Core

The VelocityResponseWriter is still a contrib component in Solr 1.4.x. In order to use it with the core distributions the following steps need to be followed:

The following jars need to be copied from contrib/velocity/src/main/solr/lib/ to $SOLR_HOME/lib:

• apache-solr-velocity-1.4-dev.jar
• velocity-1.6.1.jar
• velocity-tools-2.0-beta3.jar
• commons-beanutils-1.7.0.jar
• commons-collections-3.2.1.jar

The VelocityResponseWriter uses a more recent version of the commons lang jar than the current version of Solr core, so the jar commons-lang-2.4.jar from .../contrib/velocity/src/main/solr/lib/
should replace $SOLR_HOME/lib/commons-lang-2.1.jar

Add some configuration for this ResponseWriter to
solrconfig.xml like this:

    <queryResponseWriter name="velocity" class="org.apache.solr.request.VelocityResponseWriter"/>

Set up a RequestHandler in
solrconfig.xml:

  <requestHandler name="/itas" class="solr.SearchHandler">
     <lst name="defaults">
       <str name="v.template">browse</str>
       <str name="v.properties">velocity.properties</str>
       <str name="v.contentType">text/html;charset=UTF-8</str>
       <str name="title">Solritas</str>
 
       <str name="wt">velocity</str>
       <str name="defType">dismax</str>
       <str name="q.alt">*:*</str>
       <str name="rows">10</str>
       <str name="fl">*,score</str>
       <str name="facet">on</str>
       <str name="facet.field">title</str>
       <str name="facet.mincount">1</str>
       <str name="qf">
          text^0.5 title^1.5
       </str>
     </lst>
     <!--<lst name="invariants">-->
       <!--<str name="v.base_dir">/solr/contrib/velocity/src/main/templates</str>-->
     <!--</lst>-->
  </requestHandler>

Copy the .../contrib/velocity/src/main/solr/conf/velocity directory to $SOLR_HOME/conf/. This directory contains the Velocity templates that will be needed by the VelocityResponseWriter, and
also a style sheet, main.css. The templates and style sheet can be edited to customize the display.

Instructions to use, Solr 4.0+

These steps will get you up and running for the examples below:

• Check out Solr trunk: svn co http://svn.apache.org/viewvc/lucene/dev/branches/branch_4x/

• Build Solr: ant clean example
• Fire up Solr: cd example; java -jar start.jar
• Index sample docs: cd example/exampledocs; java -jar post.jar *.xml
• Hit the examples below...

Sample Usage

http://localhost:8983/solr/browse

• Renders browse.vm from conf/velocity. Faceted navigation included.

http://localhost:8983/solr/browse?v.template.header=Custom%20Header

• Renders browse.vm, but overrides the header.vm from conf/velocity with the specified value.

http://localhost:8983/solr/browse?debugQuery=true

• Renders browse.vm, adding in explanation views per hit, and a Velocity context dump at the end.

Options (All Versions)

• v.template: template name to use, without the .vm suffix. If not specified, "default"[.vm] will be used.

• v.template.<name>: overrides a file system template

• debugQuery: if true, default view displays explanations for each hit and additional debugging information in the footer.

• v.json: Escapes and wraps Velocity generated response with v.json parameter as a JavaScript function.

• v.layout: Template name that wraps main template (v.template). Main template renders to a $content that can be used in layout template.

• v.base_dir: overwrites default template load path (conf/velocity/).

• v.properties: specifies a Velocity properties file to be applied, found using the Solr resource loader mechanism. If not specified, no .properties file is loaded. Example: v.properties=velocity.properties
  where velocity.properties can be found using Solr's resource loader mechanism, for example in the conf/ directory (not conf/velocity which is for templates only). The .properties file could also be located inside a JAR in the lib/ directory, or other locations.

• v.contentType: sets the value of the HTTP response's Content-Type header (in case (x)html pages should be UTF-8 (instead of ISO-8859-1) encoded, make sure you set this option to text/xml;charset=UTF-8 (for
  XHTML) and text/html;charset=UTF-8 (for HTML), respectively)

Velocity Context

• esc: a Velocity EscapeTool instance

• date: a Velocity ComparisonDateTool instance

• list: a Velocity ListTool instance

• math: a Velocity MathTool instance

• number: a Velocity NumberTool instance

• page: a PageTool instance.
  page only is added to the context when response is a QueryResponse.

• request: a SolrQueryRequest

• response: a QueryResponse most
  of the time, but in some cases where QueryResponse doesn't like the request handlers
  output (AnalysisRequestHandler, for example, causes a ClassCastException parsing
  "response") the response will be a SolrResponseBase object.

• sort: a Velocity SortTool instance

TODO

• Ajax suggest
• Integrate/adapt to SolrJS
• Tie in SIMILE Timeline and SIMILE Exhibit
• Add links in default footer to this wiki page, the Solr request as XML format, and SOLR-620
• Fix multi-valued fields issue, and fl parameter usage.
• Work on "dist" target so this works easily with a nightly build.
• Make Velocity tools and engine configuration pluggable

二、Velocity文件定位过程

1、根据上述分析，首先定位layout.xml

<html>
<head>
  #parse("head.vm")
</head>
  <body>
    <div id="admin"><a href="#url_root/#/#core_name">Solr Admin</a></div>
    <div id="header">
      #parse("header.vm")
    </div>
    <div id="tabs">
      #parse("tabs.vm")
    </div>
    <div id="content">
      $content
    </div>
    <div id="footer">
      #parse("footer.vm")
    </div>
  </body>
</html>

2、其中content的内容即为browse.vm

<div class="pagination">
  #parse("pagination_top.vm")
</div>
 
## Show Error Message, if any
<div class="error">
  #parse("error.vm")
</div>
 
## Render Results, actual matching docs
<div class="results">
  #parse("results_list.vm")
</div>
 
<div class="pagination">
  #parse("pagination_bottom.vm")
</div>

3、browse.vm中最主要的搜索结果为results_list.vm

#**
 *  Render the main Results List
 *#
 
## Usually displayed inside <div class="results">
 
#if($response.response.get('grouped'))
 
  #foreach($grouping in $response.response.get('grouped'))
    #parse("hit_grouped.vm")
  #end
 
#else
 
  #foreach($doc in $response.results)
    #parse("hit.vm")
    ## Can get an extremely simple view of the doc
    ## which might be nicer for debugging
    ##parse("hit_plain.vm")
  #end
 
#end

一般情况下使用hit.vm作呈现，它对页面作了一些美工。

在某些情况下，如debug的时候，就使用hit_plain.vm进行呈现，此时将所有的属性呈现出来。

二者的对比效果如下：

4、先查看hit_plain.vm

#**
 *  An extremely plain / debug version of hit.vm
 *#
 
<table>
  ## For each field
  #foreach( $fieldName in $doc.fieldNames )
    ## For each value
    #foreach( $value in $doc.getFieldValues($fieldName) )
      <tr>
        ## Field Name
        <th align="right" valign="top">
          #if( $foreach.count == 1 )
            $fieldName:
          #end
        </th>
        ## Field Value(s)
        <td align="left" valign="top">
          $esc.html($value) <br/>
        </td>
      </tr>
    #end     ## end for each value
  #end       ## end for each field
</table>
<hr/>

就是将属性名与属性值呈现出来。

5、再看看hit.vm

#**
 *  Called for each matching document but then
 *  calls one of product_doc, join_doc or richtext_doc
 *  depending on which fields the doc has
 *#
 
#set($docId = $doc.getFieldValue('id'))
 
<div class="result-document">
 
  ## Has a "name" field ?
  #if($doc.getFieldValue('name'))
    #parse("product_doc.vm")
 
  ## Has a "compName_s" field ?
  #elseif($doc.getFieldValue('compName_s'))
    #parse("join_doc.vm")
 
  ## Fallback to richtext_doc
  #else
    #parse("richtext_doc.vm")
 
  #end
 
</div>

6、可以直接看richtest_doc.vm

#**
 *  Render a complex document in the results list
 *#
 
## Load Mime-Type List and Mapping
#parse('mime_type_lists.vm')
## Sets:
## * supportedMimeTypes, AKA supportedtypes
## * mimeExtensionsMap, AKA extMap
 
## Title
#if($doc.getFieldValue('title'))
  #set($title = $esc.html($doc.getFirstValue('title')))
#else
  #set($title = "["+$doc.getFieldValue('id')+"]")
#end
 
## URL
#if($doc.getFieldValue('url'))
  #set($url = $doc.getFieldValue('url'))
#elseif($doc.getFieldValue('resourcename'))
  #set($url = "file:///$doc.getFieldValue('resourcename')")
#else
  #set($url = "$doc.getFieldValue('id')")
#end
 
## Sort out Mime-Type
#set($ct = $list.get($doc.getFirstValue('content_type').split(";"),0))
#set($filename = $doc.getFieldValue('resourcename'))
#set($filetype = false)
#set($filetype = $mimeExtensionsMap.get($ct))
 
## TODO: falling back to file extension is convenient,
## except when you don't have an icon for that extension
## example "application/vnd.openxmlformats-officedocument.wordprocessingml.document"
## document with a .docx extension.
## It'd be nice to fall back to an "unknown" or the existing "file" type
## We sort of do this below, but only if the filename has no extension
## (anything after the last dot).
 
#if(!$filetype)
  #set($filetype = $filename.substring($filename.lastIndexOf(".")).substring(1))
#end
 
## #if(!$filetype)
##   #set($filetype = "file")
## #end
## #if(!$supportedMimeTypes.contains($filetype))
##   #set($filetype = "file")
## #end
 
## Row 1: Icon and Title and mlt link
<div class="result-title">
  ## Icon
  ## Small file type icons from http://www.splitbrain.org/projects/file_icons (public domain)
  <img src="#{url_root}/img/filetypes/${filetype}.png" align="center">
 
  ## Title, hyperlinked
  <a href="${url}" target="_blank">
    <b>$title</b></a>
 
  ## Link for MLT / More Like This / Find Similar
  <span class="mlt">
    #if($params.getBool('mlt', false) == false)
      <a href="#lensNoQ&q=id:%22$docId%22&mlt=true">
        More Like This</a>
    #end
  </span>
 
</div>
 
## Row 2?: ID / URL
<div>
  #Id: #field('id')
  ##自己修改
  ##Time: #field('tstamp')
</div>
 
## Resource Name
<div>
  #if($doc.getFieldValue('resourcename'))
    Resource name: $filename
  #elseif($url)
    URL: $url
  #end
  #if($ct)
    ($ct)
  #end
</div>
 
## Author
#if($doc.getFieldValue('author'))
  <div>
    Author: #field('author')
  </div>
#end
 
## Last_Modified Date
#if($doc.getFieldValue('last_modified'))
  <div>
    last-modified:
    #field('last_modified')
  </div>
#end
 
## Main content of doc
<div class="result-body">
  #field('content')
</div>
 
## Display Similar Documents / MLT = More Like This
<div class="mlt">
  #set($mlt = $mltResults.get($docId))
  #set($mltOn = $params.getBool('mlt'))
  #if($mltOn == true)
    <div class="field-name">
      Similar Items
    </div>
  #end
  ## If has MLT enabled An Entries to show
  #if ($mltOn && $mlt && $mlt.size() > 0)
    <ul>
      #foreach($mltHit in $mlt)
        #set($mltId = $mltHit.getFieldValue('id'))
        <li>
          <div>
            <a href="#url_for_home?q=id:$mltId">
              $mltId</a>
          </div>
          <div>
            <span class="field-name">
              Title:
            </span>
            $mltHit.getFieldValue('title')
          </div>
          <div>
            <span class="field-name">
              Author:
            </span>
            $mltHit.getFieldValue('author')
            <span class="field-name">
              Description:
            </span>
            $mltHit.getFieldValue('description')
          </div>
        </li>
      #end    ## end for each mltHit in $mlt
    </ul>
  ## Else MLT Enabled but no mlt results for this query
  #elseif($mltOn && $mlt.size() == 0)
    <div>No Similar Items Found</div>
  #end
</div>  ## div class=mlt
 
#parse('debug.vm')

由于本文件是example自带的呈现文件，其属性也按照自带的schemal.xml定义，并不适用于nutch的schema。

因此，若要改变呈现的内容，可以直接修改此文件。如将

## Row 2?: ID / URL
<div>
  #Id: #field('id')
</div>

改为：

## Row 2?: ID / URL
<div>
  ##自己修改
  #Time: #field('tstamp')
</div>

则在页面不再显示id，而是显示时间。