SQL Server性能计数器收集汇总方案（Reporting Service）

通过收集计数器信息，并将计数器信息汇总为不同粒度存储，以Reporting Service报表服务器显示。以下是计数器收集汇总的基本架构。

笔者需要收集的SQL Server计数器包括：SQL Server计数器属性详解

一、SQL Server计数器基本架构图

1、 生产服务器通过ODBC开发数据库互联ODBC配置访问监控监控服务器的SQLPerfData，将数据写入该数据库。详情请见计数器部署 SQL Server性能计数器部署（批量）

2、 通过作业调用存储过程spb_Perf_CounterDataCollect将数据从SQLPerfData写入SQLPerfDataStat数据库。

 /****************************** 功能描述：<性能计数器15秒统计>
 *   创建者：<HuangCH〉
 *   创建日期：<2014-09-22>
 *   备注说明：<根据具体业务而定，每分钟一次>
 ##########
 Change Log
 ##########
 Date                 Changer             Description
 --------------------------------------------------
 <2014-09-22>        <HuangCH>            <新建>
 --------------------------------------------------
 ***************************/
 ALTER PROC [dbo].[spb_Perf_CounterDataCollect]
 as
 --定义当前时间
 SET NOCOUNT ON
 DECLARE @FifteenSec DATETIME
         ,@OneMinute DATETIME
         ,@FiveMinute DATETIME
         ,@HalfHour DATETIME
         ,@OneHour DATETIME
         ,@SixHour DATETIME
         ,@OldFifteenSec DATETIME
 
 DECLARE @MaxRecordIndex INT
 DECLARE @MinRecordIndex INT
 
 SET @FifteenSec=CONVERT(VARCHAR(23),GETDATE(),120)
 SET @OneMinute=CONVERT(VARCHAR(16),@FifteenSec,120)
 SET @FiveMinute=Dateadd(mi,Datepart(mi,@OneMinute) / 5 * 5 - Datepart(mi,@OneMinute),@OneMinute)
 SET @HalfHour=Dateadd(mi,Datepart(mi,@OneMinute) / 30 * 30 - Datepart(mi,@OneMinute),@OneMinute)
 SET @OneHour=CONVERT(VARCHAR(14),@FifteenSec,120)+'00:00'
 SET @SixHour=Dateadd(HH,Datepart(mi,@OneHour) / 30 * 30 - Datepart(mi,@OneHour),@OneHour)
 
 ------进入循环收集处理-----------------------------------------------
 
 DECLARE @MachineName SYSNAME
 DECLARE CUR_COUNTERDATA CURSOR FOR
 SELECT DISTINCT  MachineName
 FROM [dbo].[CounterDetails_Collect] WITH(NOLOCK)
 OPEN CUR_COUNTERDATA
 FETCH NEXT FROM CUR_COUNTERDATA INTO @MachineName
 
 WHILE @@FETCH_STATUS=0
 BEGIN
 
     IF NOT EXISTS (SELECT TOP 1 1 FROM [dbo].[CounterDetails_Dts] WHERE [MachineName]=@MachineName)
     BEGIN
         --添加未插入的数据
 
         INSERT INTO [dbo].[CounterDetails_Dts]
         SELECT @MachineName,0,@FifteenSec,@OneMinute,@FiveMinute,@HalfHour,@OneHour,@SixHour,@OneMinute,@OneMinute,@FifteenSec 
 
         FETCH NEXT FROM CUR_COUNTERDATA INTO @MachineName
         CONTINUE
     END 
 
 SELECT @MinRecordIndex=[LastRecordIndex],@OldFifteenSec=LastFifteenSec  FROM [dbo].[CounterDetails_Dts] WITH(NOLOCK)
 WHERE [MachineName]=@MachineName
 
 IF DATEDIFF(MI,@OldFifteenSec,@FifteenSec)>25
 BEGIN
   SET  @MinRecordIndex=0
 END
 
 SELECT @MaxRecordIndex=MAX([RecordIndex]),@FifteenSec=MAX(CONVERT(DATETIME,LEFT(A.CounterDateTime,23)))
 FROM [SQLPerfData].[dbo].[CounterData]  A WITH(NOLOCK)
 WHERE CounterID=(SELECT TOP 1 CounterID FROM [dbo].[CounterDetails_Collect] B WITH(NOLOCK) WHERE B.MachineName=@MachineName)
 AND [RecordIndex]>@MinRecordIndex
 
 --若由于重启导致index恢复，则要重新计算
 IF @MaxRecordIndex IS NULL
 BEGIN
 
 SET @MaxRecordIndex=@MinRecordIndex--上次最大值
 SELECT @MinRecordIndex=NumberOfRecords FROM --当前最大值
 (
 SELECT ROW_NUMBER() OVER(ORDER BY LogStartTime DESC) ROW_ID,NumberOfRecords
 FROM [SQLPerfData].[dbo].[DisplayToID]
 WHERE GUID IN(
 SELECT DISTINCT A.GUID FROM [SQLPerfData].[dbo].[CounterData]  A WITH(NOLOCK)
 JOIN [dbo].[CounterDetails_Collect] B WITH(NOLOCK)  ON A.CounterID=B.CounterID
 WHERE   B.MachineName=@MachineName)
 ) AA
 WHERE ROW_ID=1
 
 --删除已经收集的
 DELETE A FROM [SQLPerfData].[dbo].[CounterData]  A WITH(NOLOCK)
 JOIN [dbo].[CounterDetails_Collect] B WITH(NOLOCK)  ON A.CounterID=B.CounterID
 WHERE A.[RecordIndex]>@MinRecordIndex AND A.[RecordIndex]<=@MaxRecordIndex AND  B.MachineName=@MachineName
 
 --恢复当前有效值
 SET @MaxRecordIndex=@MinRecordIndex
 SET @MinRecordIndex=0
 END
 
 IF @MaxRecordIndex IS NOT NULL
 BEGIN
 BEGIN TRY
     --BEGIN TRAN
         ----收集
         INSERT INTO [dbo].[CounterData_FifteenSeconds](CounterID,CounterDateTime,CounterValue)
         SELECT A.CounterID,CONVERT(DATETIME,LEFT(A.CounterDateTime,23)),A.CounterValue
         FROM [SQLPerfData].[dbo].[CounterData]  A WITH(NOLOCK)
         JOIN [dbo].[CounterDetails_Collect] B WITH(NOLOCK)  ON A.CounterID=B.CounterID
         WHERE A.[RecordIndex]>@MinRecordIndex AND A.[RecordIndex]<=@MaxRecordIndex AND  B.MachineName=@MachineName
 
         UPDATE  [dbo].[CounterDetails_Dts]
         SET LastFifteenSec=@FifteenSec,[LastRecordIndex]=@MaxRecordIndex
         WHERE MachineName=@MachineName
         --COMMIT
 END TRY
 BEGIN CATCH
     IF @@TRANCOUNT>0
        COMMIT;
     --THROW
 END CATCH
 
 --更新
 /*
 UPDATE  [dbo].[CounterDetails_Dts]
 SET LastFifteenSec=@FifteenSec,[LastRecordIndex]=@MaxRecordIndex
 WHERE [CounterID]=@CounterID */
 
 --删除30分钟内的数据
 
 END
 
 DELETE A
 FROM [CounterData_FifteenSeconds]   A
 JOIN [dbo].[CounterDetails_Collect] B WITH(NOLOCK)  ON A.CounterID=B.CounterID
 WHERE B.MachineName=@MachineName AND CounterDateTime<DATEADD(WEEK,-1,@FifteenSec)
 
 FETCH NEXT FROM CUR_COUNTERDATA INTO @MachineName
 END
 CLOSE CUR_COUNTERDATA
 DEALLOCATE CUR_COUNTERDATA
 
 --监控

spb_Perf_CounterDataCollect

3、 汇总取均值，将15秒钟数据汇总成1分钟，5分钟，半小时，一小时，六小时数据。各个粒度的数据用于不同时段的报表显示。例如1分钟数据汇总可用于显示一天内的报表，一个月的报表则需要一小时的数据才能正常显示。通过作业调用spb_Perf_CounterDataDts汇总。

 /****************************** 功能描述：<性能计数器15秒统计>
 *   创建者：<HuangCH〉
 *   创建日期：<2014-09-22>
 *   备注说明：<根据具体业务而定，每分钟一次>
 ##########
 Change Log
 ##########
 Date                 Changer             Description
 --------------------------------------------------
 <2014-09-22>        <HuangCH>            <新建>
 --------------------------------------------------
 ***************************/
 ALTER PROC [dbo].[spb_Perf_CounterDataDts]
 as
 --定义当前时间
 SET NOCOUNT ON
 DECLARE @OneMinute DATETIME
         ,@FiveMinute DATETIME
         ,@HalfHour DATETIME
         ,@OneHour DATETIME
         ,@SixHour DATETIME
 DECLARE @OldOneMinute  datetime
         ,@OldFiveMinute datetime
         ,@OldHalfHour datetime
         ,@OldOneHour datetime
         ,@OldSixHour datetime
         ,@OldFifteenSec DATETIME
 
 --MachineName
 DECLARE @MachineName SYSNAME
 
 DECLARE CUR_CounterDataDts CURSOR FOR
 
 SELECT DISTINCT  MachineName
 FROM [dbo].[CounterDetails_Dts] WITH(NOLOCK)
 OPEN CUR_CounterDataDts
 FETCH NEXT FROM CUR_CounterDataDts INTO @MachineName
 
 WHILE @@FETCH_STATUS=0
     BEGIN
         SELECT
         @OldFifteenSec= LastFifteenSec,
         @OldOneMinute= LastOneMinute,
         @OldFiveMinute= LastFiveMinute,
         @OldHalfHour=LastHalfHour,
         @OldOneHour=LastOneHour,
         @OldSixHour=LastSixHour
         FROM [dbo].[CounterDetails_Dts] WITH(NOLOCK)
         WHERE MachineName=@MachineName
 
         SET @OneMinute=CONVERT(VARCHAR(16),@OldFifteenSec,120)
         SET @FiveMinute=Dateadd(mi,Datepart(mi,@OneMinute) / 5 * 5 - Datepart(mi,@OneMinute),@OneMinute)
         SET @HalfHour=Dateadd(mi,Datepart(mi,@OneMinute) / 30 * 30 - Datepart(mi,@OneMinute),@OneMinute)
         SET @OneHour=CONVERT(VARCHAR(14),@OneMinute,120)+'00:00'
         SET @SixHour=Dateadd(HH,Datepart(HH,@OneHour) / 6 * 6 -Datepart(hh,@OneHour),@OneHour)
 
         --超过60分钟未收集，则只收集60分钟以内的数据
         IF DATEDIFF(MI,@OldOneMinute,@OneMinute)>60
         BEGIN
           SET @OldOneMinute=DATEADD(MI,-60,@OneMinute)
         END
 
         UPDATE [dbo].[CounterDetails_Dts]
         SET LastOneMinute=@OneMinute,LastFiveMinute=@FiveMinute,LastHalfHour=@HalfHour,LastOneHour=@OneHour,LastSixHour=@SixHour
         WHERE MachineName=@MachineName
 
         BEGIN TRY
             --一分钟
             INSERT INTO [dbo].[CounterData_OneMinute]
             SELECT A.CounterID,CONVERT(VARCHAR(16),A.CounterDateTime,120),AVG(A.CounterValue)
             FROM [dbo].[CounterData_FifteenSeconds]  A WITH(NOLOCK)
             JOIN [dbo].[CounterDetails_Collect] B WITH(NOLOCK) ON A.CounterID=B.CounterID
             WHERE B.MachineName=@MachineName AND A.CounterDateTime <= @OneMinute And A.CounterDateTime > @OldOneMinute
             GROUP BY A.CounterID,CONVERT(VARCHAR(16),A.CounterDateTime,120)
 
         END TRY
         BEGIN CATCH
             IF @@ROWCOUNT>0
               COMMIT
         END CATCH
         BEGIN TRY
             --五分钟
             INSERT INTO [dbo].[CounterData_FiveMinute]
             SELECT A.CounterID,Dateadd(mi,Datepart(mi,A.CounterDateTime) / 5 * 5 - Datepart(mi,A.CounterDateTime),CONVERT(DATETIME,CONVERT(CHAR(16),A.CounterDateTime,120))),AVG(A.CounterValue)
             FROM [dbo].[CounterData_OneMinute]  A WITH(NOLOCK)
             JOIN [dbo].[CounterDetails_Collect] B WITH(NOLOCK) ON A.CounterID=B.CounterID
             WHERE B.MachineName=@MachineName AND A.CounterDateTime <= @FiveMinute And A.CounterDateTime > @OldFiveMinute
             GROUP BY A.CounterID,Dateadd(mi,Datepart(mi,A.CounterDateTime) / 5 * 5 - Datepart(mi,A.CounterDateTime),CONVERT(DATETIME,CONVERT(CHAR(16),A.CounterDateTime,120)))
 
         END TRY
         BEGIN CATCH
             IF @@ROWCOUNT>0
               COMMIT
 
         END CATCH
         BEGIN TRY
             --半个小时
             INSERT INTO [dbo].[CounterData_HalfHour]
             SELECT A.CounterID,Dateadd(mi,Datepart(mi,A.CounterDateTime) / 30 * 30 - Datepart(mi,A.CounterDateTime),CONVERT(DATETIME,CONVERT(CHAR(16),A.CounterDateTime,120))),AVG(A.CounterValue)
             FROM [dbo].[CounterData_OneMinute]  A WITH(NOLOCK)
             JOIN [dbo].[CounterDetails_Collect] B WITH(NOLOCK) ON A.CounterID=B.CounterID
             WHERE B.MachineName=@MachineName AND A.CounterDateTime <= @HalfHour And A.CounterDateTime > @OldHalfHour
             GROUP BY A.CounterID,Dateadd(mi,Datepart(mi,A.CounterDateTime) / 30 * 30 - Datepart(mi,A.CounterDateTime),CONVERT(DATETIME,CONVERT(CHAR(16),A.CounterDateTime,120)))
 
         END TRY
         BEGIN CATCH
             IF @@ROWCOUNT>0
               COMMIT
         END CATCH
 
         BEGIN TRY
             --一小时
             INSERT INTO [dbo].[CounterData_OneHour]
             SELECT A.CounterID,CONVERT(VARCHAR(14),A.CounterDateTime,120)+'00:00',AVG(A.CounterValue)
             FROM [dbo].[CounterData_OneMinute]  A WITH(NOLOCK)
             JOIN [dbo].[CounterDetails_Collect] B WITH(NOLOCK) ON A.CounterID=B.CounterID
             WHERE B.MachineName=@MachineName AND A.CounterDateTime <= @OneHour And A.CounterDateTime > @OldOneHour
             GROUP BY A.CounterID,CONVERT(VARCHAR(14),A.CounterDateTime,120)
 
         END TRY
         BEGIN CATCH
             IF @@ROWCOUNT>0
               COMMIT
         END CATCH            
 
         BEGIN TRY
             --六小时
             INSERT INTO [dbo].[CounterData_SixHour]
             SELECT A.CounterID,Dateadd(HH,Datepart(HH,A.CounterDateTime) / 6 * 6 -Datepart(hh,A.CounterDateTime),A.CounterDateTime),AVG(A.CounterValue)
             FROM [dbo].[CounterData_OneMinute]  A WITH(NOLOCK)
             JOIN [dbo].[CounterDetails_Collect] B WITH(NOLOCK) ON A.CounterID=B.CounterID
             WHERE B.MachineName=@MachineName AND A.CounterDateTime <= @SixHour And A.CounterDateTime > @OldSixHour
             GROUP BY A.CounterID,Dateadd(HH,Datepart(HH,A.CounterDateTime) / 6 * 6 -Datepart(hh,A.CounterDateTime),A.CounterDateTime)
         END TRY
         BEGIN CATCH
             IF @@ROWCOUNT>0
               COMMIT
         END CATCH
     FETCH NEXT FROM CUR_CounterDataDts INTO @MachineName
     END
 CLOSE CUR_CounterDataDts
 DEALLOCATE CUR_CounterDataDts

spb_Perf_CounterDataDts

4、 计算历史数据为基线数据，与当前数据对比。SQL Server基线算法（同比和环比）

二、SQL Server汇总统计架构

如上图所示，计数器架构分别存储三个数据库SQLPerfData、SQLPerfDataStat、SQLPerfDataStat_Histroy；这三个计数器的功能具体如下：

1、 SQLPerfData接收计数器信息

DisplayToID：计数器自动生成，记录计数器启动信息

CounterDetails：计数器自动生成，记录计数器的基本信息

CounterData：记录计数器传入的数据

2、 SQLPerfDataStat统计计数器信息

基本信息模块：

CounterDetails_Collect：用于同步 CounterDetails表数据，用于轮训统计

CounterDetails_Collect_bak：无用计数器备份

CounterDetails_Dts：统计传输控制表，以服务器为单位，记录服务器统计传输时间。

CounterDetails_Report：同步CounterDetails_Collect表信息，通过优化，用于报表显示作用。

数据模块：

CounterData_FifteenSeconds：15秒数据收集，用于同步CounterData表数据。

CounterData_OneMinute：一分钟数据收集，15秒数据均值计算而成；适合查看当天数据。保留7天数据。

CounterData_FiveMinute：五分钟数据收集，一分钟数据均值计算而成；适合查看4天内数据。保留7天数据。

CounterData_HalfHour：30分钟数据收集，一分钟数据均值计算而成；适合查看一周内数据。保留7天数据。

CounterData_OneHour：一小时数据收集，一分钟数据均值计算而成；适合查看一个月内数据。保留永久

CounterData_SixHour：六小时数据收集，一分钟数据均值计算而成；适合查看一年内数据。保留永久

基线模块：

CBaseCounterData_OneMinute：同比基线，可以计算未来一周的数据走势。保留7天数据。

RBaseCounterData_OneMinute：环比基线，可以计算未来一天的数据走势。保留7天数据。

手动维护模块：

CounterTypeDetails：计数器类型和描述信息记录表。

MonitorContorl：手动维护，计数器报警监控控制表。

3、SQLPerfDataStat_Histroy归档计数器信息

CounterData_OneMinute_bak：一分钟数据备份，保留永久。

CounterData_FiveMinute_bak：五分钟数据备份，保留永久。

CounterData_HalfHour_bak：30分钟数据备份，保留永久。

基线模块：

CBaseCounterData_OneMinute_bak：同比基线备份，保留7天。

RBaseCounterData_OneMinute_bak：环比基线备份，保留7天。