DEVOPS技术实践_05：sonar静态代码扫描

一、SonarQube静态代码扫描平台

1.1 安装

https://www.sonarqube.org/官网

1.2 下载软件包

https://www.sonarqube.org/downloads/

[root@sonar-server ~]# mkdir /usr/local/sonarc

[root@sonar-server ~]# cd /usr/local/sonarc

[root@sonar-server sonarc]# wget https://binaries.sonarsource.com/Distribution/sonarqube/sonarqube-6.7.7.zip

[root@sonar-server sonarc]# wget https://binaries.sonarsource.com/Distribution/sonar-scanner-cli/sonar-scanner-cli-3.3.0.1492-linux.zip

[root@sonar-server sonarc]# ll

-rw-r--r--.  root root  Apr    sonarqube-6.7..zip
-rw-r--r--.  root root   Jan   : sonar-scanner-cli-3.3.0.1492-linux.zip

sonar是一个用于代码质量管理的开放平台。通过插件机制，sonar可以集成不同的测试工具，代码分析工具，以及持续集成工具。比如pmd-cpd，checkstyle，findbugs。Jkens。通过不同的插件对这些结果进行再加工处理。通过量化的方式度量代码质量的变化，从而可以方便的对不同规模和种类的工程进行代码质量管理。同时sonar还对大量的集成工具提供了接口支持，可以很方便的在持续集成中使用Sonar。

[root@sonar-server sonarc]# yum -y install java-1.8.0-openjdk-devel.x86_64

官方文档

环境要求

https://docs.sonarqube.org/latest/requirements/requirements/

JDK  JRE：1.8

[root@sonar-server sonarc]# java -version

openjdk version "1.8.0_201"
OpenJDK Runtime Environment (build 1.8.0_201-b09)
OpenJDK -Bit Server VM (build 25.201-b09, mixed mode)

1.3 安装mysql 5.6 +

[root@sonar-server sonarc]# mkdir /usr/local/mysql

[root@sonar-server sonarc]# cd /usr/local/mysql

[root@sonar-server mysql]# wget http://dev.mysql.com/get/mysql-community-release-el7-5.noarch.rpm

[root@sonar-server mysql]# rpm -ivh mysql-community-release-el7-5.noarch.rpm

Preparing...                          ################################# [%]
Updating / installing...
   :mysql-community-release-el7-    ################################# [%]

[root@sonar-server mysql]# yum -y install mysql  mysql-devel  mysql-server mysql-utilities

Installed:
  mysql-community-client.x86_64 :5.6.-.el7
  mysql-community-devel.x86_64 :5.6.-.el7
  mysql-community-libs.x86_64 :5.6.-.el7
  mysql-community-server.x86_64 :5.6.-.el7
  mysql-utilities.noarch :1.6.-.el7  

启动mysql

[root@sonar-server mysql]# systemctl start mysqld

[root@sonar-server mysql]# netstat -ntlp

tcp6              :::                 :::*                    LISTEN      /mysqld  

1.4 创建数据库

[root@sonar-server mysql]# mysql -u root

mysql> show databases;

+--------------------+
| Database           |
+--------------------+
| information_schema |
| mysql              |
| performance_schema |
+--------------------+

mysql> create database sonar default CHARSET utf8;

mysql> show databases;

+--------------------+
| Database           |
+--------------------+
| information_schema |
| mysql              |
| performance_schema |
| sonar              |
+--------------------+

mysql> grant all on sonar.*  to sonar@'%'  IDENTIFIED by 'meiyoumima';

mysql> flush PRIVILEGES;

1.5 安装sonar

If you're running on Linux, you must ensure that:
vm.max_map_count is greater or equals to
fs.file-max is greater or equals to
the user running SonarQube can open at least  file descriptors
the user running SonarQube can open at least  threads
You can see the values with the following commands:
sysctl vm.max_map_count
sysctl fs.file-max
ulimit -n
ulimit -u

根据文档执行命令

[root@sonar-server ~]# sysctl vm.max_map_count
vm.max_map_count =
[root@sonar-server ~]# sysctl fs.file-max
fs.file-max =
[root@sonar-server ~]# ulimit -n

[root@sonar-server ~]# ulimit -u

修改这些参数
[root@sonar-server ~]# sysctl -w vm.max_map_count=
vm.max_map_count =
[root@sonar-server ~]# sysctl -w fs.file-max=
fs.file-max =
[root@sonar-server ~]# ulimit -n
[root@sonar-server ~]# ulimit -u 

添加sonar用户（sonar明确规定不允许使用root运行）

[root@sonar-server ~]# useradd sonar

[root@sonar-server ~]# cd /usr/local/sonarc/

[root@sonar-server sonarc]# unzip sonarqube-6.7.7.zip  -d /home/sonar/

[root@sonar-server sonarc]# cd /home/sonar/

[root@sonar-server sonar]# mv sonarqube-6.7.7 sonarqube

修改sonar的配置文件

[root@sonar-server sonar]# cd sonarqube

[root@sonar-server sonarqube]# ll

drwxr-xr-x.  root root   Apr    bin
drwxr-xr-x.  root root    Apr    conf
-rw-r--r--.  root root  Apr    COPYING
drwxr-xr-x.  root root    Apr    data
drwxr-xr-x.  root root   Apr    elasticsearch
drwxr-xr-x.  root root    Apr    extensions
drwxr-xr-x.  root root   Apr    lib
drwxr-xr-x.  root root     Apr    logs
drwxr-xr-x.  root root    Apr    temp
drwxr-xr-x.  root root  Apr    web

[root@sonar-server sonarqube]# ll ./bin/

drwxr-xr-x.  root root   Apr    jsw-license
drwxr-xr-x.  root root   Apr    linux-x86-
drwxr-xr-x.  root root   Apr    linux-x86-
drwxr-xr-x.  root root   Apr    macosx-universal-
drwxr-xr-x.  root root  Apr    windows-x86-
drwxr-xr-x.  root root  Apr    windows-x86-

[root@sonar-server sonarqube]# ll ./bin/linux-x86-64/

drwxr-xr-x.  root root      Apr    lib
-rwxr-xr-x.  root root   Apr    sonar.sh     #sonar的启动脚本
-rwxr-xr-x.  root root  Apr    wrapper

[root@sonar-server sonarqube]# vim conf/sonar.properties

sonar.web.host=172.25.254.133
sonar.web.port=
sonar.jdbc.username=sonar
sonar.jdbc.password=meiyoumima
sonar.jdbc.url=jdbc:mysql://localhost:3306/sonar?useUnicode=true&characterEncoding=utf8&rewriteBatchedStatements=true&useConfigs=maxPerformance&useSSL=false

修改文件的属主属组

[root@sonar-server sonar]# chown -R sonar:sonar  /home/sonar/*

切换用户设置环境变量

[root@sonar-server sonar]# su - sonar

[sonar@sonar-server ~]$ vim .bash_profile

export SONAR_HOME=/home/sonar/sonarqube
export PATH=$PATH:$SONAR_HOME/bin

[sonar@sonar-server ~]$ source .bash_profile

[sonar@sonar-server ~]$ ./sonarqube/bin/linux-x86-64/sonar.sh start

Starting SonarQube...
Started SonarQube.

1.7 启动sonar

[sonar@sonar-server ~]$ ./sonarqube/bin/linux-x86-64/sonar.sh

Usage: ./sonarqube/bin/linux-x86-/sonar.sh { console | start | stop | restart | status | dump }

[sonar@sonar-server ~]$ ./sonarqube/bin/linux-x86-64/sonar.sh console

Running SonarQube...     #输出信息
wrapper  | --> Wrapper Started as Console
wrapper  | Launching a JVM...
jvm     | Wrapper (Version 3.2.) http://wrapper.tanukisoftware.org
jvm     |   Copyright - Tanuki Software, Inc.  All Rights Reserved.
jvm     |
jvm     | 2019.04. :: INFO  app[][o.s.a.AppFileSystem] Cleaning or creating temp directory /home/sonar/sonarqube/temp
jvm     | 2019.04. :: INFO  app[][o.s.a.es.EsSettings] Elasticsearch listening on /127.0.0.1:
jvm     | 2019.04. :: INFO  app[][o.s.a.p.ProcessLauncherImpl] Launch process[[key='es', ipcIndex=, logFilenamePrefix=es]] from [/home/sonar/sonarqube/elasticsearch]: /home/sonar/sonarqube/elasticsearch/bin/elasticsearch -Epath.conf=/home/sonar/sonarqube/temp/conf/es
jvm     | 2019.04. :: INFO  app[][o.s.a.SchedulerImpl] Waiting for Elasticsearch to be up and running
jvm     | 2019.04. :: INFO  app[][o.e.p.PluginsService] no modules loaded
jvm     | 2019.04. :: INFO  app[][o.e.p.PluginsService] loaded plugin [org.elasticsearch.transport.Netty4Plugin]
jvm     | 2019.04. :: INFO  app[][o.s.a.SchedulerImpl] Process[es] is up
jvm     | 2019.04. :: INFO  app[][o.s.a.p.ProcessLauncherImpl] Launch process[[key='web', ipcIndex=, logFilenamePrefix=web]] from [/home/sonar/sonarqube]: /usr/lib/jvm/java-1.8.-openjdk-1.8.0.201.b09-.el7_6.x86_64/jre/bin/java -Djava.awt.headless=true -Dfile.encoding=UTF- -Djava.io.tmpdir=/home/sonar/sonarqube/temp -Xmx512m -Xms128m -XX:+HeapDumpOnOutOfMemoryError -cp ./lib/common/*:./lib/server/*:/home/sonar/sonarqube/lib/jdbc/mysql/mysql-connector-java-5.1.42.jar org.sonar.server.app.WebServer /home/sonar/sonarqube/temp/sq-process7782197132334949329properties

看日志

[sonar@sonar-server ~]$ cd sonarqube/logs/
[sonar@sonar-server logs]$ tail -f es.log
2019.04. :: INFO  es[][o.e.n.Node] closing ...
2019.04. :: INFO  es[][o.e.n.Node] closed
[sonar@sonar-server logs]$ tail -f sonar.log
2019.04. :: INFO  app[][o.s.a.SchedulerImpl] Process [es] is stopped
2019.04. :: INFO  app[][o.s.a.SchedulerImpl] SonarQube is stopped
<-- Wrapper Stopped
[sonar@sonar-server logs]$ tail -n  web.log
    at org.sonar.db.DefaultDatabase.checkConnection(DefaultDatabase.java:)    #数据库问题，不能使用sonar@localhost
    ...  common frames omitted
Caused by: java.sql.SQLException: Access denied for user 'sonar'@'localhost' (using password: YES)

尝试使用sonar连接数据库

[sonar@sonar-server logs]$ mysql -hlocalhost -usonar -pmeiyoumima
Warning: Using a password on the command line interface can be insecure.
ERROR  (): Access denied for user 'sonar'@'localhost' (using password: YES)
[sonar@sonar-server logs]$ mysql -h127.0.0. -usonar -pmeiyoumima
Warning: Using a password on the command line interface can be insecure.
ERROR  (): Access denied for user 'sonar'@'localhost' (using password: YES)
[sonar@sonar-server logs]$ mysql -h172.25.254. -usonar -pmeiyoumima    #成功
mysql> show databases;
+--------------------+
| Database |
+--------------------+
| information_schema |
| sonar |
+--------------------+

修改配置文件连接数据库

[sonar@sonar-server logs]$ vim /home/sonar/sonarqube/conf/sonar.properties

sonar.web.host=172.25.254.133
sonar.web.port=
sonar.jdbc.username=sonar
sonar.jdbc.password=meiyoumima
sonar.jdbc.url=jdbc:mysql://172.25.254.133:3306/sonar?useUnicode=true&characterEncoding=utf8&rewriteBatchedStatements=true&useConfigs=maxPerformance&useSSL=false

[sonar@sonar-server logs]$ /home/sonar/sonarqube/bin/linux-x86-64/sonar.sh start

[sonar@sonar-server logs]$ /home/sonar/sonarqube/bin/linux-x86-64/sonar.sh console

Running SonarQube...
SonarQube is already running.

成功

1.8 浏览器访问

初始密码：admin  admin登录

1.9 安装一个maketpalce的汉化插件

重启

二、配置snoar-scanner

2.1 安装

[sonar@sonar-server logs]$  unzip /usr/local/sonarc/sonar-scanner-cli-3.3.0.1492-linux.zip  -d /home/sonar/

[sonar@sonar-server logs]$ cd

[sonar@sonar-server ~]$ ll

drwxr-xr-x.  sonar sonar  Apr    sonarqube
drwxr-xr-x.   sonar sonar   Jan   : sonar-scanner-3.3.0.1492-linux

[sonar@sonar-server ~]$ mv sonar-scanner-3.3.0.1492-linux sonar-scanner

[sonar@sonar-server ~]$ cd sonar-scanner/

[sonar@sonar-server sonar-scanner]$ ll

drwxr-xr-x.  sonar sonar   Jan   : bin
drwxr-xr-x.  sonar sonar   Jan   : conf
drwxr-xr-x.  sonar sonar  Jan   : jre
drwxr-xr-x.  sonar sonar   Jan   : lib

[sonar@sonar-server sonar-scanner]$ vim conf/sonar-scanner.properties

sonar.host.url=http://172.25.254.133:9000
sonar.sourceEncoding=UTF-

[root@sonar-server ~]# vim .bash_profile

export SCAN_HOME=/home/sonar/sonar-scanner/
export PATH=$PATH:$SCAN_HOME/bin

[root@sonar-server ~]# source .bash_profile

2.2 测试

[root@sonar-server ~]# sonar-scanner -X

[root@sonar-server ~]# mkdir code

[root@sonar-server ~]# cd code/

[root@sonar-server code]# vim sonar-project.properties

sonar.projectKey=test-project1
sonar.projectName=cloud
sonar.projectVersion=1.0
sonar.source=src
sonar.language=python
sonar.sourceEncoding=UTF-

[root@sonar-server code]mkdir src

[root@sonar-server code]# vim  ./src/test.py

print("HelloWorld")
print("HelloWorld")
print("HelloWorld")
print("HelloWorld")
print("HelloWorld")
print("HelloWorld")
print("HelloWorld")
print("HelloWorld")

执行

[root@sonar-server code]# python ./src/test.py
HelloWorld
HelloWorld
HelloWorld
HelloWorld
HelloWorld
HelloWorld
HelloWorld
HelloWorld

[root@sonar-server ~]# sonar-scanner

ERROR: You must define the following mandatory properties for 'Unknown': sonar.projectKey, sonar.sources
ERROR:
ERROR: Re-run SonarQube Scanner using the -X switch to enable full debug logging.

2.3 修改权限

[root@sonar-server code]# chown sonar:sonar /root/code/* -R

[root@sonar-server code]# ll

-rw-r--r--.  sonar sonar  Apr  : sonar-project.properties

[root@sonar-server ~]# sonar-scanner

ERROR: Error during SonarQube Scanner execution
ERROR: No quality profiles have been found, you probably don't have any language plugin installed.
ERROR:
ERROR: Re-run SonarQube Scanner using the -X switch to enable full debug logging.

2.4 安装plugin

[root@sonar-server code]# sonar-scanner

ERROR: Error during SonarQube Scanner execution
ERROR: You must install a plugin that supports the language 'python'
ERROR:     #依然报错
ERROR: Re-run SonarQube Scanner using the -X switch to enable full debug logging.

把python修改为py

[root@sonar-server code]# vim sonar-project.properties

sonar.projectKey=test-project1
sonar.projectName=cloud
sonar.projectVersion=1.0
sonar.language=py     #查阅修改此参数   https://www.cnblogs.com/ckat/p/3638887.html
sonar.sources=src
sonar.sourceEncoding=UTF-

[root@sonar-server code]# sonar-scanner

INFO: Scanner configuration file: /home/sonar/sonar-scanner-3.3.0.1492-linux/conf/sonar-scanner.properties
INFO: Project root configuration file: /root/code/sonar-project.properties
INFO: SonarQube Scanner 3.3.0.1492
INFO: Java 1.8.0_121 Oracle Corporation (-bit)
INFO: Linux 3.10.-.el7.x86_64 amd64
INFO: User cache: /root/.sonar/cache
INFO: SonarQube server 6.7.
INFO: Default locale: "en_US", source code encoding: "UTF-8"
INFO: Publish mode
INFO: Load global settings
INFO: Load global settings (done) | time=62ms
INFO: Server id: A623D34D-AWoVn6_8P1KovjAYWYot
INFO: User cache: /root/.sonar/cache
INFO: Load plugins index
INFO: Load plugins index (done) | time=112ms
INFO: Plugin [l10nzh] defines 'l10nen' as base plugin. This metadata can be removed from manifest of l10n plugins since version 5.2.
INFO: Process project properties
INFO: Load project repositories
INFO: Load project repositories (done) | time=35ms
INFO: Load quality profiles
INFO: Load quality profiles (done) | time=29ms
INFO: Load active rules
INFO: Load active rules (done) | time=344ms
INFO: Load metrics repository
INFO: Load metrics repository (done) | time=26ms
INFO: Project key: test-project1
INFO: -------------  Scan cloud
INFO: Load server rules
INFO: Load server rules (done) | time=34ms
INFO: Base dir: /root/code
INFO: Working dir: /root/code/.scannerwork
INFO: Source paths: src
INFO: Source encoding: UTF-, default locale: en_US
INFO: Language is forced to py
INFO: Index files
INFO:  file indexed
INFO: Quality profile for py: Sonar way
INFO: Sensor Python Squid Sensor [python]
INFO: Sensor Python Squid Sensor [python] (done) | time=101ms
INFO: Sensor Cobertura Sensor for Python coverage [python]
INFO: Sensor Cobertura Sensor for Python coverage [python] (done) | time=11ms
INFO: Sensor PythonXUnitSensor [python]
INFO: Sensor PythonXUnitSensor [python] (done) | time=0ms
INFO: Sensor Zero Coverage Sensor
INFO: Sensor Zero Coverage Sensor (done) | time=28ms
INFO: Sensor CPD Block Indexer
INFO: Sensor CPD Block Indexer (done) | time=0ms
INFO: SCM Publisher is disabled
INFO:  file had no CPD blocks
INFO: Calculating CPD for  files
INFO: CPD calculation finished
INFO: Analysis report generated in 274ms, dir size= KB
INFO: Analysis reports compressed in 15ms, zip size= KB
INFO: Analysis report uploaded in 410ms
INFO: ANALYSIS SUCCESSFUL, you can browse http://172.25.254.133:9000/dashboard/index/test-project1
INFO: Note that you will be able to access the updated dashboard once the server has processed the submitted analysis report
INFO: More about the report processing at http://172.25.254.133:9000/api/ce/task?id=AWo_l71TD8zrSa_Nq7yJ
INFO: Task total time: 2.315 s
INFO: ------------------------------------------------------------------------
INFO: EXECUTION SUCCESS
INFO: ------------------------------------------------------------------------
INFO: Total time: .566s
INFO: Final Memory: 9M/183M
INFO: ------------------------------------------------------------------------

成功

2.5 浏览器检查结果

2.6 sonarqube配置完善

配置强制登陆

添加两个用户

2.7 对用户做权限设置

更改项目类型

配置-->项目--->管理

所有项目改称私有

权限管理

创建组-->用户加入组--->权限模板--->应用权限模板

用户test01添加进组

创建权限模板

添加组

创建一个test1开的项目

换test1用户登陆，就能看到那个项目

顺利完成！！！