基于MIPI的高性能成像系统

硬件组件
Digilent Genesys ZU × 1 （FPGA平台） Digilent PCAM5 × 1 （MIPI摄像头）

软件组件
AMD-Xilinx Vivado 设计套件

介绍
从简单的嵌入式视觉到自动驾驶汽车和无人机，图像处理是许多应用的核心。Xilinx MPSoC 具有内置的 DisplayPort（DP） 功能，并在可编程逻辑 IO 中支持 MIPI DPhy，从这两方面看，这是一个出色的嵌入式视觉平台。

同时使用人工智能和机器学习功能使我们能够创建一个复杂的嵌入式视觉处理系统。

在这个项目中，我们将探索使用 PCAM（FMC扩展板） 和 Display Port 建立和运行图像处理。然后，我们可以添加图像处理 IP 内核以进一步展示FPGA的处理图像的能力。

可编程逻辑设计（PL端设计）
在我们的设计中，我们将包括以下 IP

Zynq MPSoC Processing System - 启用 DisplayPort、I2C 和 GPIO EMIO

MIPI CSI2 RX Sub System - 从 PCAM 接收 MIPI CSI2 流。

Sensor Demosaic - 将 RAW 像素格式转换为 RGB 像素格式。

Gamma LUT - gamma图像校正

VDMA - 将映像写入PS中的 DDR 内存

Video Timing Generator - 生成输出视频时序

AXIS to Video Out - 从 AXI 流转换为视频流

Clock Wizards - 用于生成视频像素时钟和 MIPI CSI2 参考时钟

除了 IP，我们还需要考虑时钟架构，对于这个解决方案，我们设计了以下时钟树：

AXI 时钟 - 150 MHz - 这为 AXI Stream 和 AXI lite 接口提供时钟

DPHY 参考时钟 - 200 MHz - 由Clock Wizards生成

Pixel Clock - 74.25 Mhz - 用于 1280 x 720 60 FPS - 由Clock Wizards生成

完成的框图应如下所示，内核配置如下所示。

为了控制和配置 PCAM5（FMC-MIPI扩展板），我们使用 I2C 和 GPIO，GPIO 信号用于启用 PCAM5 并为其供电。而 I2C 用于配置 PCAM5 本身。

我们可以使用EMIO GPIO将处理器 GPIO 扩展到可编程逻辑。虽然 I2C 由 PS IIC0 提供，但是我们需要在应用程序软件中配置器启用。

要启用 MPSoC 中的显示端口，我们需要在 I/O 配置页面下配置 DisplayPort 外设。硬件上使用了 Bank 505 的MGT。

我们还需要配置通过 EMIO 连接到 PL 的辅助引脚。这意味着我们需要以不同于 PS 中的 MIO 的方式处理它们，辅助输出使能引脚低电平有效，因此我们需要在使用 EMIO 时对信号使用反相器。

配置好 DisplayPort 后，我们就可以启用 PL 的实时视频输入。这在 PS-PL 配置下可用。

最后配置是设置 GPIO 以提供一个 1 位的 EMIO，这样我们就可以打开和关闭 PCAM5。

Sensor将通过 I2C 进行配置，最后通过两个 MIPI 通道以 280 Mbps 的数据速率输出 10 位 RAW 视频。

因此，我们需要配置 MIPI CSI-2 RX 子系统

由于我们只有一个 MIPI 接口，我们将配置 MIPI 内核以包含所有共享逻辑。如果我们在同一个 bank 上有多个 MIPI 接口，则下一个 MIPI 接口将被配置为使用示例设计中的共享逻辑。

MIPI 接口的最终设置是为 MIPI 通道和时钟选择 IO bank 和引脚分配。正如我们在这里定义的那样，我们不需要将它们添加到定义引脚位置的 XDC 文件中。

创建可用图像的下一步是转换原始图像，其中包含有关每个像素一个颜色通道的信息。

在包含红色、绿色和蓝色元素的图像中，这称为去拜耳化，由 Sensor Demosaic IP 块实现。

对于我们的应用程序，PCAM5 将以以下格式输出像素 BGBG/GRGR

绿色是红色和蓝色的两倍，因为我们的眼睛对绿色更敏感，如果可见光谱位于中间，则绿色比位于两端的红色和蓝色更敏感。

处理的下一步是实现Gamma校正 IP 内核

最后一个阶段是插入一个VDMA，这将使视频数据能够从 PS DDR 写入到内存中。

为了向显示端口提供输出视频的时序，我们在发生器模式下使用VTC。

最后阶段是 AXIS 到视频输出 IP，这将采用视频时序控制器、时序信号和 AXI 视频流来创建具有正确时序的输出视频。

其输出将连接到显示端口视频实时输入。

这需要一点思考，实时视频输入具有 36 位视频输入，像素的每个元素为 12 位。我们正在使用每个元素 10 位，这使其整体为 30 位。

通过填充 LSB，到视频输出的 AXIS 流将按比例放大输出块，从每像素 30 位到每像素 36 位。

确保 AXIS 到 Video Out IP 内核输出的颜色通道与 DisplayPort 实时视频输入正确对齐。最简单的方法是在 AXI Stream 中使用 AXIS 子集转换器根据需要切换颜色通道。

为了帮助调试和理解设计，我还在设计中包含了几个集成逻辑分析仪。

随着设计的完成，我们现在可以构建硬件设计并使用 SDK 实现软件应用程序。

软件开发
完成硬件设计后，下一步是编写将在可编程逻辑中配置 IP 的软件。此配置将允许他们通过视频，可以显示在 DisplayPort 监视器上。

因此，我们的应用软件将执行以下步骤

配置 GPIO 并启用 PCAM5 电源

将 Video Mode 1280 x 720 设置为 60 FPS 并配置VTC

3）配置VDMA帧大小和内存存储

配置 Senso Demosaic

配置伽玛校正

使用 I2C 链路配置 PCAM5

启用从 VDMA 读取和写入帧缓冲区

设置显示端口

就像我们处理其他项目一样，我们需要导入硬件规范、创建应用程序和 BSP。

为了能够在显示端口上使用实时视频源，我们需要首先更新 BSP 设置并重新生成它。

在 BSP 设置中将 avbuf 更改为 dppsu 并等待 BSP 重新生成。

正确配置 BSP 后，我们可以创建应用程序代码。BSP 将提供使用 PL 中的 IP 块所需的所有 API 函数。

主要应用如下

#include <stdio.h>
#include "platform.h"
#include "xil_printf.h"
#include "xvtc.h"
#include "vga_modes.h"
#include "xv_tpg.h"
#include "xvidc.h"
#include "xavbuf.h"
#include "xavbuf_clk.h"
#include "xvidc.h"
#include "xdpdma_video_example.h"
#include "xiicps.h"
#include "i2c.h"
#include "xaxivdma.h"
#include "xaxivdma_i.h"
#include "xgpiops.h"
#include "xv_demosaic.h"
#include "xv_gamma_lut.h"
#include "math.h"
XVtc VtcInst;
XVtc_Config *vtc_config ;
XV_tpg tpg;
XV_tpg_Config *tpg_config;
XIicPs iic_cam;
XAxiVdma vdma;
XAxiVdma_DmaSetup vdmaDMA;
XAxiVdma_Config *vdmaConfig;
XGpioPs gp_cam;
VideoMode video;
XV_demosaic cfa;
XV_gamma_lut gamma_inst;
u8 SendBuffer [10];
u8 RecvBuffer [10];
u16 gamma_reg[1024];
#define DEMO_MAX_FRAME (720*1280)
#define DEMO_STRIDE (1280*2)
#define DISPLAY_NUM_FRAMES 3
#define IIC_cam XPAR_XIICPS_0_DEVICE_ID
#define cam_gpio XPAR_XGPIOPS_0_DEVICE_ID
#define CAM_ID 0x78
#define IIC_CAM_ADDR 0x3c
#define IIC_SCLK_RATE 100000
#define MUX_ADDR 0x70
void detect_camera();
int Initial_setting_1 ( u32 *cfg_init , int cfg_init_QTY );
void read_camera();
void gamma_calc(float gamma_val);
u32 frameBuf[DISPLAY_NUM_FRAMES][DEMO_MAX_FRAME];
u32 *pFrames[DISPLAY_NUM_FRAMES];
int main()
{
XVtc_Timing vtcTiming;
XVtc_SourceSelect SourceSelect;
XGpioPs_Config *GPIO_Config;
int Status;
init_platform();
disable_caches();
print("Hello World\n\r");
vtc_config = XVtc_LookupConfig(XPAR_VTC_0_DEVICE_ID);
XVtc_CfgInitialize(&VtcInst, vtc_config, vtc_config->BaseAddress);
GPIO_Config = XGpioPs_LookupConfig(cam_gpio);
Status= XGpioPs_CfgInitialize(&gp_cam,GPIO_Config,GPIO_Config->BaseAddr);
XGpioPs_SetOutputEnablePin(&gp_cam,78,1);
XGpioPs_SetDirectionPin(&gp_cam,78,1);
XGpioPs_WritePin(&gp_cam,78,0x0);
usleep(2000000);
XGpioPs_WritePin(&gp_cam,78,0x1);
video = VMODE_1280x720;
vtcTiming.HActiveVideo = video.width; /**< Horizontal Active Video Size */
vtcTiming.HFrontPorch = video.hps - video.width; /**< Horizontal Front Porch Size */
vtcTiming.HSyncWidth = video.hpe - video.hps; /**< Horizontal Sync Width */
vtcTiming.HBackPorch = video.hmax - video.hpe + 1; /**< Horizontal Back Porch Size */
vtcTiming.HSyncPolarity = video.hpol; /**< Horizontal Sync Polarity */
vtcTiming.VActiveVideo = video.height; /**< Vertical Active Video Size */
vtcTiming.V0FrontPorch = video.vps - video.height; /**< Vertical Front Porch Size */
vtcTiming.V0SyncWidth = video.vpe - video.vps; /**< Vertical Sync Width */
vtcTiming.V0BackPorch = video.vmax - video.vpe + 1;; /**< Horizontal Back Porch Size */
vtcTiming.V1FrontPorch = video.vps - video.height; /**< Vertical Front Porch Size */
vtcTiming.V1SyncWidth = video.vpe - video.vps; /**< Vertical Sync Width */
vtcTiming.V1BackPorch = video.vmax - video.vpe + 1;; /**< Horizontal Back Porch Size */
vtcTiming.VSyncPolarity = video.vpol; /**< Vertical Sync Polarity */
vtcTiming.Interlaced = 0;
memset((void *)&SourceSelect, 0, sizeof(SourceSelect));
SourceSelect.VBlankPolSrc = 1;
SourceSelect.VSyncPolSrc = 1;
SourceSelect.HBlankPolSrc = 1;
SourceSelect.HSyncPolSrc = 1;
SourceSelect.ActiveVideoPolSrc = 1;
SourceSelect.ActiveChromaPolSrc= 1;
SourceSelect.VChromaSrc = 1;
SourceSelect.VActiveSrc = 1;
SourceSelect.VBackPorchSrc = 1;
SourceSelect.VSyncSrc = 1;
SourceSelect.VFrontPorchSrc = 1;
SourceSelect.VTotalSrc = 1;
SourceSelect.HActiveSrc = 1;
SourceSelect.HBackPorchSrc = 1;
SourceSelect.HSyncSrc = 1;
SourceSelect.HFrontPorchSrc = 1;
SourceSelect.HTotalSrc = 1;
XVtc_RegUpdateEnable(&VtcInst);
XVtc_SetGeneratorTiming(&VtcInst, &vtcTiming);
XVtc_SetSource(&VtcInst, &SourceSelect);
XVtc_EnableGenerator(&VtcInst);
XVtc_Enable(&VtcInst);
setup_tpg();
for (int i = 0; i < 3; i++)
{
pFrames[i] = frameBuf[i];
}
vdmaConfig = XAxiVdma_LookupConfig(XPAR_AXIVDMA_0_DEVICE_ID);
XAxiVdma_CfgInitialize(&vdma, vdmaConfig, vdmaConfig->BaseAddress);
//video = VMODE_1280x720;
vdmaDMA.FrameDelay = 0;
vdmaDMA.EnableCircularBuf = 1;
vdmaDMA.EnableSync = 0;
vdmaDMA.PointNum = 0;
vdmaDMA.EnableFrameCounter = 0;
vdmaDMA.VertSizeInput = video.height;
vdmaDMA.HoriSizeInput = (video.width)*4;
vdmaDMA.FixedFrameStoreAddr = 0;
vdmaDMA.FrameStoreStartAddr[0] = (u32) pFrames[0];
vdmaDMA.Stride = (video.width)*4;
XAxiVdma_DmaConfig(&vdma, XAXIVDMA_WRITE, &(vdmaDMA));
Status = XAxiVdma_DmaSetBufferAddr(&vdma, XAXIVDMA_WRITE,vdmaDMA.FrameStoreStartAddr);
XAxiVdma_DmaConfig(&vdma, XAXIVDMA_READ, &(vdmaDMA));
XAxiVdma_DmaSetBufferAddr(&vdma, XAXIVDMA_READ,vdmaDMA.FrameStoreStartAddr);
XV_demosaic_Initialize(&cfa, XPAR_V_DEMOSAIC_0_DEVICE_ID);
XV_demosaic_Set_HwReg_width(&cfa, video.width);
XV_demosaic_Set_HwReg_height(&cfa, video.height);
XV_demosaic_Set_HwReg_bayer_phase(&cfa, 0x03);
XV_demosaic_EnableAutoRestart(&cfa);
XV_demosaic_Start(&cfa);
gamma_calc(1.6);
XV_gamma_lut_Initialize(&gamma_inst, XPAR_V_GAMMA_LUT_0_DEVICE_ID);
XV_gamma_lut_Set_HwReg_width(&gamma_inst, video.width);
XV_gamma_lut_Set_HwReg_height(&gamma_inst, video.height);
XV_gamma_lut_Set_HwReg_video_format(&gamma_inst, 0x00);
XV_gamma_lut_Write_HwReg_gamma_lut_0_Bytes(&gamma_inst, 0,(int *) gamma_reg, 2048);
XV_gamma_lut_Write_HwReg_gamma_lut_1_Bytes(&gamma_inst, 0,(int *) gamma_reg, 2048);
XV_gamma_lut_Write_HwReg_gamma_lut_2_Bytes(&gamma_inst, 0,(int *) gamma_reg, 2048);
XV_gamma_lut_Start(&gamma_inst);
XV_gamma_lut_EnableAutoRestart(&gamma_inst);
detect_camera();
SendBuffer[0]= 0x31;
SendBuffer[1]= 0x03;
SendBuffer[2]= 0x11;
Status = XIicPs_MasterSendPolled(&iic_cam, SendBuffer, 3, IIC_CAM_ADDR);
//writeReg(0x3103, 0x11);
//[7]=1 Software reset; [6]=0 Software power down; Default=0x02
SendBuffer[0]= 0x30;
SendBuffer[1]= 0x08;
SendBuffer[2]= 0x82;
Status = XIicPs_MasterSendPolled(&iic_cam, SendBuffer, 3, IIC_CAM_ADDR);
//writeReg(0x3008, 0x82);
usleep(1000000);
Initial_setting_1 ( cfg_init , 63 );
Initial_setting_1 ( cfg_simple_awb, 19 );
Initial_setting_1 ( cfg_720p_60fps , 38 );
xil_printf("Configuration Complete\n\r");
Status = XAxiVdma_DmaStart(&vdma, XAXIVDMA_WRITE);
Status = XAxiVdma_StartParking(&vdma, 0, XAXIVDMA_WRITE);
XAxiVdma_DmaStart(&vdma, XAXIVDMA_READ);
XAxiVdma_StartParking(&vdma, 0, XAXIVDMA_READ);
run_dppsu();
while(1){
}
cleanup_platform();
return 0;
}
void gamma_calc(float gamma_val)
{
int i;
for(i = 0; i<1024; i++){
gamma_reg[i] = (pow((i / 1024.0), (1/gamma_val)) * 1024.0);
}
}
void setup_tpg()
{
u32 height,width,status;
tpg_config = XV_tpg_LookupConfig(XPAR_XV_TPG_0_DEVICE_ID);
XV_tpg_CfgInitialize(&tpg, tpg_config, tpg_config->BaseAddress);
#ifdef DEBUG
status = XV_tpg_IsReady(&tpg);
printf("TPG Status %u \n\r", (unsigned int) status);
#endif
XV_tpg_Set_height(&tpg, (u32) video.height);
XV_tpg_Set_width(&tpg, (u32) video.width);
height = XV_tpg_Get_height(&tpg);
width = XV_tpg_Get_width(&tpg);
XV_tpg_Set_colorFormat(&tpg,XVIDC_CSF_RGB);
XV_tpg_Set_maskId(&tpg, 0x0);
XV_tpg_Set_motionSpeed(&tpg, 0x4);
#ifdef DEBUG
printf("info from tpg %u %u \n\r", (unsigned int)height, (unsigned int)width);
#endif
XV_tpg_Set_bckgndId(&tpg,XTPG_BKGND_SOLID_BLUE); //XTPG_BKGND_TARTAN_COLOR_BARS);
#ifdef DEBUG
status = XV_tpg_Get_bckgndId(&tpg);
printf("Status %x \n\r", (unsigned int) status);
#endif
XV_tpg_EnableAutoRestart(&tpg);
XV_tpg_Start(&tpg);
#ifdef DEBUG
status = XV_tpg_IsIdle(&tpg);
printf("Status %u \n\r", (unsigned int) status);
#endif
}
void detect_camera()
{
XIicPs_Config *iic_conf;
u32 Status;
iic_conf = XIicPs_LookupConfig(IIC_cam);
XIicPs_CfgInitialize(&iic_cam,iic_conf,iic_conf->BaseAddress);
XIicPs_SetSClk(&iic_cam, IIC_SCLK_RATE);
SendBuffer[0]= 0x01;
SendBuffer[1]= 0x00;
Status = XIicPs_MasterSendPolled(&iic_cam, SendBuffer, 1, MUX_ADDR);
if (Status != XST_SUCCESS) {
print("SW I2C write error\n\r");
return XST_FAILURE;
}
usleep(2000000);
SendBuffer[0]= 0x31;
SendBuffer[1]= 0x00;
Status = XIicPs_MasterSendPolled(&iic_cam, SendBuffer, 2, IIC_CAM_ADDR);
if (Status != XST_SUCCESS) {
print("I2C write error\n\r");
return XST_FAILURE;
}
Status = XIicPs_MasterRecvPolled(&iic_cam, RecvBuffer,1, IIC_CAM_ADDR);
if (Status != XST_SUCCESS) {
print("I2C read error\n\r");
return XST_FAILURE;
}
if(RecvBuffer[0] != CAM_ID ){
print("Camera not detected\n\r");
}
else{
print("Camera detected \n\r");
}
}
void read_camera()
{
XIicPs_Config *iic_conf;
u32 Status;
iic_conf = XIicPs_LookupConfig(IIC_cam);
XIicPs_CfgInitialize(&iic_cam,iic_conf,iic_conf->BaseAddress);
XIicPs_SetSClk(&iic_cam, IIC_SCLK_RATE);
SendBuffer[0]= 0x30;
SendBuffer[1]= 0x0e;
Status = XIicPs_MasterSendPolled(&iic_cam, SendBuffer, 2, IIC_CAM_ADDR);
if (Status != XST_SUCCESS) {
print("I2C write error\n\r");
return XST_FAILURE;
}
Status = XIicPs_MasterRecvPolled(&iic_cam, RecvBuffer,1, IIC_CAM_ADDR);
if (Status != XST_SUCCESS) {
print("I2C read error\n\r");
return XST_FAILURE;
}
}
int Initial_setting_1 ( u32 *cfg_init , int cfg_init_QTY )
{
s32 Status , byte_count;
int i ;
u8 SendBuffer[10];
for(i=0;i<(cfg_init_QTY*2);i+=2){
SendBuffer[1]= *(cfg_init + i);
SendBuffer[0]= (*(cfg_init + i))>>8;
SendBuffer[2]= *(cfg_init + i + 1);
Status = XIicPs_MasterSendPolled(&iic_cam, SendBuffer, 3, IIC_CAM_ADDR);
if (Status != XST_SUCCESS) {
print("I2C read error\n\r");
return XST_FAILURE;
}
usleep(1000);
}
return XST_SUCCESS;
}
具有 I2C 配置的文件

//config_word_t const cfg_init_[] =
static u32 cfg_init [][2] =
{
//[7]=0 Software reset; [6]=1 Software power down; Default=0x02
{0x3008, 0x42},
//[1]=1 System input clock from PLL; Default read = 0x11
{0x3103, 0x03},
//[3:0]=0000 MD2P,MD2N,MCP,MCN input; Default=0x00
{0x3017, 0x00},
//[7:2]=000000 MD1P,MD1N, D3:0 input; Default=0x00
{0x3018, 0x00},
//[6:4]=001 PLL charge pump, [3:0]=1000 MIPI 8-bit mode
{0x3034, 0x18},
//PLL1 configuration
//[7:4]=0001 System clock divider /1, [3:0]=0001 Scale divider for MIPI /1
{0x3035, 0x11},
//[7:0]=56 PLL multiplier
{0x3036, 0x38},
//[4]=1 PLL root divider /2, [3:0]=1 PLL pre-divider /1
{0x3037, 0x11},
//[5:4]=00 PCLK root divider /1, [3:2]=00 SCLK2x root divider /1, [1:0]=01 SCLK root divider /2
{0x3108, 0x01},
//PLL2 configuration
//[5:4]=01 PRE_DIV_SP /1.5, [2]=1 R_DIV_SP /1, [1:0]=00 DIV12_SP /1
{0x303D, 0x10},
//[4:0]=11001 PLL2 multiplier DIV_CNT5B = 25
{0x303B, 0x19},
{0x3630, 0x2e},
{0x3631, 0x0e},
{0x3632, 0xe2},
{0x3633, 0x23},
{0x3621, 0xe0},
{0x3704, 0xa0},
{0x3703, 0x5a},
{0x3715, 0x78},
{0x3717, 0x01},
{0x370b, 0x60},
{0x3705, 0x1a},
{0x3905, 0x02},
{0x3906, 0x10},
{0x3901, 0x0a},
{0x3731, 0x02},
//VCM debug mode
{0x3600, 0x37},
{0x3601, 0x33},
//System control register changing not recommended
{0x302d, 0x60},
//??
{0x3620, 0x52},
{0x371b, 0x20},
//?? DVP
{0x471c, 0x50},
{0x3a13, 0x43},
{0x3a18, 0x00},
{0x3a19, 0xf8},
{0x3635, 0x13},
{0x3636, 0x06},
{0x3634, 0x44},
{0x3622, 0x01},
{0x3c01, 0x34},
{0x3c04, 0x28},
{0x3c05, 0x98},
{0x3c06, 0x00},
{0x3c07, 0x08},
{0x3c08, 0x00},
{0x3c09, 0x1c},
{0x3c0a, 0x9c},
{0x3c0b, 0x40},
//[7]=1 color bar enable, [3:2]=00 eight color bar
{0x503d, 0x00},
//[2]=1 ISP vflip, [1]=1 sensor vflip
{0x3820, 0x46},
//[7:5]=010 Two lane mode, [4]=0 MIPI HS TX no power down, [3]=0 MIPI LP RX no power down, [2]=1 MIPI enable, [1:0]=10 Debug mode; Default=0x58
{0x300e, 0x45},
//[5]=0 Clock free running, [4]=1 Send line short packet, [3]=0 Use lane1 as default, [2]=1 MIPI bus LP11 when no packet; Default=0x04
{0x4800, 0x14},
{0x302e, 0x08},
//[7:4]=0x3 YUV422, [3:0]=0x0 YUYV
//{0x4300, 0x30},
//[7:4]=0x6 RGB565, [3:0]=0x0 {b[4:0],g[5:3],g[2:0],r[4:0]}
{0x4300, 0x6f},
{0x501f, 0x01},
{0x4713, 0x03},
{0x4407, 0x04},
{0x440e, 0x00},
{0x460b, 0x35},
//[1]=0 DVP PCLK divider manual control by 0x3824[4:0]
{0x460c, 0x20},
//[4:0]=1 SCALE_DIV=INT(3824[4:0]/2)
{0x3824, 0x01},
//MIPI timing
// {0x4805, 0x10}, //LPX global timing select=auto
// {0x4818, 0x00}, //hs_prepare + hs_zero_min ns
// {0x4819, 0x96},
// {0x482A, 0x00}, //hs_prepare + hs_zero_min UI
//
// {0x4824, 0x00}, //lpx_p_min ns
// {0x4825, 0x32},
// {0x4830, 0x00}, //lpx_p_min UI
//
// {0x4826, 0x00}, //hs_prepare_min ns
// {0x4827, 0x32},
// {0x4831, 0x00}, //hs_prepare_min UI
//[7]=1 LENC correction enabled, [5]=1 RAW gamma enabled, [2]=1 Black pixel cancellation enabled, [1]=1 White pixel cancellation enabled, [0]=1 Color interpolation enabled
{0x5000, 0x07},
//[7]=0 Special digital effects, [5]=0 scaling, [2]=0 UV average disabled, [1]=1 Color matrix enabled, [0]=1 Auto white balance enabled
{0x5001, 0x03}
};
static u32 cfg_simple_awb[][2] =
{
// Disable Advanced AWB
{0x518d ,0x00},
{0x518f ,0x20},
{0x518e ,0x00},
{0x5190 ,0x20},
{0x518b ,0x00},
{0x518c ,0x00},
{0x5187 ,0x10},
{0x5188 ,0x10},
{0x5189 ,0x40},
{0x518a ,0x40},
{0x5186 ,0x10},
{0x5181 ,0x58},
{0x5184 ,0x25},
{0x5182 ,0x11},
// Enable simple AWB
{0x3406 ,0x00},
{0x5183 ,0x80},
{0x5191 ,0xff},
{0x5192 ,0x00},
{0x5001 ,0x03}
};
static u32 cfg_720p_60fps[][2] =
{//1280 x 720 binned, RAW10, MIPISCLK=280M, SCLK=56Mz, PCLK=56M
//PLL1 configuration
{0x3008, 0x42},
//[7:4]=0010 System clock divider /2, [3:0]=0001 Scale divider for MIPI /1
{0x3035, 0x21},
//[7:0]=70 PLL multiplier
{0x3036, 0x46},
//[4]=0 PLL root divider /1, [3:0]=5 PLL pre-divider /1.5
{0x3037, 0x05},
//[5:4]=01 PCLK root divider /2, [3:2]=00 SCLK2x root divider /1, [1:0]=01 SCLK root divider /2
{0x3108, 0x11},
//[6:4]=001 PLL charge pump, [3:0]=1010 MIPI 10-bit mode
{0x3034, 0x1A},
//[3:0]=0 X address start high byte
{0x3800, (0 >> 8) & 0x0F},
//[7:0]=0 X address start low byte
{0x3801, 0 & 0xFF},
//[2:0]=0 Y address start high byte
{0x3802, (8 >> 8) & 0x07},
//[7:0]=0 Y address start low byte
{0x3803, 8 & 0xFF},
//[3:0] X address end high byte
{0x3804, (2619 >> 8) & 0x0F},
//[7:0] X address end low byte
{0x3805, 2619 & 0xFF},
//[2:0] Y address end high byte
{0x3806, (1947 >> 8) & 0x07},
//[7:0] Y address end low byte
{0x3807, 1947 & 0xFF},
//[3:0]=0 timing hoffset high byte
{0x3810, (0 >> 8) & 0x0F},
//[7:0]=0 timing hoffset low byte
{0x3811, 0 & 0xFF},
//[2:0]=0 timing voffset high byte
{0x3812, (0 >> 8) & 0x07},
//[7:0]=0 timing voffset low byte
{0x3813, 0 & 0xFF},
//[3:0] Output horizontal width high byte
{0x3808, (1280 >> 8) & 0x0F},
//[7:0] Output horizontal width low byte
{0x3809, 1280 & 0xFF},
//[2:0] Output vertical height high byte
{0x380a, (720 >> 8) & 0x7F},
//[7:0] Output vertical height low byte
{0x380b, 720 & 0xFF},
//HTS line exposure time in # of pixels
{0x380c, (1896 >> 8) & 0x1F},
{0x380d, 1896 & 0xFF},
//VTS frame exposure time in # lines
{0x380e, (984 >> 8) & 0xFF},
{0x380f, 984 & 0xFF},
//[7:4]=0x3 horizontal odd subsample increment, [3:0]=0x1 horizontal even subsample increment
{0x3814, 0x31},
//[7:4]=0x3 vertical odd subsample increment, [3:0]=0x1 vertical even subsample increment
{0x3815, 0x31},
//[2]=0 ISP mirror, [1]=0 sensor mirror, [0]=1 horizontal binning
{0x3821, 0x01},
//little MIPI shit: global timing unit, period of PCLK in ns * 2(depends on # of lanes)
{0x4837, 36}, // 1/56M*2
//Undocumented anti-green settings
{0x3618, 0x00}, // Removes vertical lines appearing under bright light
{0x3612, 0x59},
{0x3708, 0x64},
{0x3709, 0x52},
{0x370c, 0x03},
//[7:4]=0x0 Formatter RAW, [3:0]=0x0 BGBG/GRGR
{0x4300, 0x00},
//[2:0]=0x3 Format select ISP RAW (DPC)
{0x501f, 0x03},
{0x3008, 0x02},
};
显示端口配置文件

/***************************** Include Files *********************************/
#include "xil_exception.h"
#include "xil_printf.h"
#include "xil_cache.h"
#include "xdpdma_video_example.h"
/************************** Constant Definitions *****************************/
#define DPPSU_DEVICE_ID XPAR_PSU_DP_DEVICE_ID
#define AVBUF_DEVICE_ID XPAR_PSU_DP_DEVICE_ID
#define DPDMA_DEVICE_ID XPAR_XDPDMA_0_DEVICE_ID
#define DPPSU_INTR_ID 151
#define DPDMA_INTR_ID 154
#define INTC_DEVICE_ID XPAR_SCUGIC_0_DEVICE_ID
#define DPPSU_BASEADDR XPAR_PSU_DP_BASEADDR
#define AVBUF_BASEADDR XPAR_PSU_DP_BASEADDR
#define DPDMA_BASEADDR XPAR_PSU_DPDMA_BASEADDR
#define BUFFERSIZE 1280 * 720 * 4 /* HTotal * VTotal * BPP */
#define LINESIZE 1280 * 4 /* HTotal * BPP */
#define STRIDE LINESIZE /* The stride value should
be aligned to 256*/
/************************** Variable Declarations ***************************/
u8 Frame[BUFFERSIZE] __attribute__ ((__aligned__(256)));
XDpDma_FrameBuffer FrameBuffer;
/**************************** Type Definitions *******************************/
/*****************************************************************************/
/**
*
* Main function to call the DPDMA Video example.
*
* @param None
*
* @return XST_SUCCESS if successful, otherwise XST_FAILURE.
*
* @note None
*
******************************************************************************/
int run_dppsu()
{
int Status;
Xil_DCacheDisable();
Xil_ICacheDisable();
xil_printf("DPDMA Generic Video Example Test \r\n");
Status = DpdmaVideoExample(&RunCfg);
if (Status != XST_SUCCESS) {
xil_printf("DPDMA Video Example Test Failed\r\n");
return XST_FAILURE;
}
xil_printf("Successfully ran DPDMA Video Example Test\r\n");
return XST_SUCCESS;
}
/*****************************************************************************/
/**
*
* The purpose of this function is to illustrate how to use the XDpDma device
* driver in Graphics overlay mode.
*
* @param RunCfgPtr is a pointer to the application configuration structure.
*
* @return XST_SUCCESS if successful, else XST_FAILURE.
*
* @note None.
*
*****************************************************************************/
int DpdmaVideoExample(Run_Config *RunCfgPtr)
{
u32 Status;
/* Initialize the application configuration */
InitRunConfig(RunCfgPtr);
Status = InitDpDmaSubsystem(RunCfgPtr);
if (Status != XST_SUCCESS) {
return XST_FAILURE;
}
SetupInterrupts(RunCfgPtr);
//xil_printf("Generating Overlay.....\n\r");
GraphicsOverlay(Frame, RunCfgPtr);
/* Populate the FrameBuffer structure with the frame attributes */
FrameBuffer.Address = (INTPTR)Frame;
FrameBuffer.Stride = STRIDE;
FrameBuffer.LineSize = LINESIZE;
FrameBuffer.Size = BUFFERSIZE;
XDpDma_DisplayGfxFrameBuffer(RunCfgPtr->DpDmaPtr, &FrameBuffer);
return XST_SUCCESS;
}
/*****************************************************************************/
/**
*
* The purpose of this function is to initialize the application configuration.
*
* @param RunCfgPtr is a pointer to the application configuration structure.
*
* @return None.
*
* @note None.
*
*****************************************************************************/
void InitRunConfig(Run_Config *RunCfgPtr)
{
/* Initial configuration parameters. */
RunCfgPtr->DpPsuPtr = &DpPsu;
RunCfgPtr->IntrPtr = &Intr;
RunCfgPtr->AVBufPtr = &AVBuf;
RunCfgPtr->DpDmaPtr = &DpDma;
RunCfgPtr->VideoMode = XVIDC_VM_1280x720_60_P;
RunCfgPtr->Bpc = XVIDC_BPC_12;//XVIDC_BPC_8;
RunCfgPtr->ColorEncode = XDPPSU_CENC_RGB;
RunCfgPtr->UseMaxCfgCaps = 1;
RunCfgPtr->LaneCount = LANE_COUNT_2;
RunCfgPtr->LinkRate = LINK_RATE_540GBPS;
RunCfgPtr->EnSynchClkMode = 0;
RunCfgPtr->UseMaxLaneCount = 1;
RunCfgPtr->UseMaxLinkRate = 1;
}
/*****************************************************************************/
/**
*
* The purpose of this function is to initialize the DP Subsystem (XDpDma,
* XAVBuf, XDpPsu)
*
* @param RunCfgPtr is a pointer to the application configuration structure.
*
* @return None.
*
* @note None.
*
*****************************************************************************/
int InitDpDmaSubsystem(Run_Config *RunCfgPtr)
{
u32 Status;
XDpPsu *DpPsuPtr = RunCfgPtr->DpPsuPtr;
XDpPsu_Config *DpPsuCfgPtr;
XAVBuf *AVBufPtr = RunCfgPtr->AVBufPtr;
XDpDma_Config *DpDmaCfgPtr;
XDpDma *DpDmaPtr = RunCfgPtr->DpDmaPtr;
/* Initialize DisplayPort driver. */
DpPsuCfgPtr = XDpPsu_LookupConfig(DPPSU_DEVICE_ID);
XDpPsu_CfgInitialize(DpPsuPtr, DpPsuCfgPtr, DpPsuCfgPtr->BaseAddr);
/* Initialize Video Pipeline driver */
XAVBuf_CfgInitialize(AVBufPtr, DpPsuPtr->Config.BaseAddr, AVBUF_DEVICE_ID);
/* Initialize the DPDMA driver */
DpDmaCfgPtr = XDpDma_LookupConfig(DPDMA_DEVICE_ID);
XDpDma_CfgInitialize(DpDmaPtr,DpDmaCfgPtr);
/* Initialize the DisplayPort TX core. */
Status = XDpPsu_InitializeTx(DpPsuPtr);
if (Status != XST_SUCCESS) {
return XST_FAILURE;
}
/* Set the format graphics frame for DPDMA*/
Status = XDpDma_SetGraphicsFormat(DpDmaPtr, RGBA8888);
if (Status != XST_SUCCESS) {
return XST_FAILURE;
}
/* Set the format graphics frame for Video Pipeline*/
Status = XAVBuf_SetInputNonLiveGraphicsFormat(AVBufPtr, RGBA8888);
if (Status != XST_SUCCESS) {
return XST_FAILURE;
}
/* Set the QOS for Video */
XDpDma_SetQOS(RunCfgPtr->DpDmaPtr, 11);
/* Enable the Buffers required by Graphics Channel */
//XAVBuf_EnableGraphicsBuffers(RunCfgPtr->AVBufPtr, 1);
XAVBuf_SetInputLiveVideoFormat(&AVBufPtr, RGB_10BPC);
XAVBuf_EnableVideoBuffers(&AVBufPtr, 1);
/* Set the output Video Format */
XAVBuf_SetOutputVideoFormat(AVBufPtr, RGB_10BPC);
/* Select the Input Video Sources.
* Here in this example we are going to demonstrate
* graphics overlay over the TPG video.
*/
XAVBuf_InputVideoSelect(AVBufPtr, XAVBUF_VIDSTREAM1_LIVE, XAVBUF_VIDSTREAM2_NONE);
/* Configure Video pipeline for graphics channel */
//XAVBuf_ConfigureGraphicsPipeline(AVBufPtr);
/* Configure the output video pipeline */
XAVBuf_ConfigureOutputVideo(AVBufPtr);
/* Disable the global alpha, since we are using the pixel based alpha */
XAVBuf_SetBlenderAlpha(AVBufPtr, 0, 0);
/* Set the clock mode */
XDpPsu_CfgMsaEnSynchClkMode(DpPsuPtr, RunCfgPtr->EnSynchClkMode);
/* Set the clock source depending on the use case.
* Here for simplicity we are using PS clock as the source*/
XAVBuf_SetAudioVideoClkSrc(AVBufPtr, XAVBUF_PL_CLK, XAVBUF_PS_CLK);
/* Issue a soft reset after selecting the input clock sources */
XAVBuf_SoftReset(AVBufPtr);
return XST_SUCCESS;
}
/*****************************************************************************/
/**
*
* The purpose of this function is to setup call back functions for the DP
* controller interrupts.
*
* @param RunCfgPtr is a pointer to the application configuration structure.
*
* @return None.
*
* @note None.
*
*****************************************************************************/
void SetupInterrupts(Run_Config *RunCfgPtr)
{
XDpPsu *DpPsuPtr = RunCfgPtr->DpPsuPtr;
XScuGic *IntrPtr = RunCfgPtr->IntrPtr;
XScuGic_Config *IntrCfgPtr;
u32 IntrMask = XDPPSU_INTR_HPD_IRQ_MASK | XDPPSU_INTR_HPD_EVENT_MASK;
XDpPsu_WriteReg(DpPsuPtr->Config.BaseAddr, XDPPSU_INTR_DIS, 0xFFFFFFFF);
XDpPsu_WriteReg(DpPsuPtr->Config.BaseAddr, XDPPSU_INTR_MASK, 0xFFFFFFFF);
XDpPsu_SetHpdEventHandler(DpPsuPtr, DpPsu_IsrHpdEvent, RunCfgPtr);
XDpPsu_SetHpdPulseHandler(DpPsuPtr, DpPsu_IsrHpdPulse, RunCfgPtr);
/* Initialize interrupt controller driver. */
IntrCfgPtr = XScuGic_LookupConfig(INTC_DEVICE_ID);
XScuGic_CfgInitialize(IntrPtr, IntrCfgPtr, IntrCfgPtr->CpuBaseAddress);
/* Register ISRs. */
XScuGic_Connect(IntrPtr, DPPSU_INTR_ID,
(Xil_InterruptHandler)XDpPsu_HpdInterruptHandler, RunCfgPtr->DpPsuPtr);
/* Trigger DP interrupts on rising edge. */
XScuGic_SetPriorityTriggerType(IntrPtr, DPPSU_INTR_ID, 0x0, 0x03);
/* Connect DPDMA Interrupt */
XScuGic_Connect(IntrPtr, DPDMA_INTR_ID,
(Xil_ExceptionHandler)XDpDma_InterruptHandler, RunCfgPtr->DpDmaPtr);
/* Initialize exceptions. */
Xil_ExceptionInit();
Xil_ExceptionRegisterHandler(XIL_EXCEPTION_ID_IRQ_INT,
(Xil_ExceptionHandler)XScuGic_DeviceInterruptHandler,
INTC_DEVICE_ID);
/* Enable exceptions for interrupts. */
Xil_ExceptionEnableMask(XIL_EXCEPTION_IRQ);
Xil_ExceptionEnable();
/* Enable DP interrupts. */
XScuGic_Enable(IntrPtr, DPPSU_INTR_ID);
XDpPsu_WriteReg(DpPsuPtr->Config.BaseAddr, XDPPSU_INTR_EN, IntrMask);
/* Enable DPDMA Interrupts */
XScuGic_Enable(IntrPtr, DPDMA_INTR_ID);
XDpDma_InterruptEnable(RunCfgPtr->DpDmaPtr, XDPDMA_IEN_VSYNC_INT_MASK);
}
/*****************************************************************************/
/**
*
* The purpose of this function is to generate a Graphics frame of the format
* RGBA8888 which generates an overlay on 1/2 of the bottom of the screen.
* This is just to illustrate the functionality of the graphics overlay.
*
* @param RunCfgPtr is a pointer to the application configuration structure.
* @param Frame is a pointer to a buffer which is going to be populated with
* rendered frame
*
* @return Returns a pointer to the frame.
*
* @note None.
*
*****************************************************************************/
u8 *GraphicsOverlay(u8* Frame, Run_Config *RunCfgPtr)
{
u64 Index;
u32 *RGBA;
RGBA = (u32 *) Frame;
/*
* Red at the top half
* Alpha = 0x0F
* */
for(Index = 0; Index < (BUFFERSIZE/4) /2; Index ++) {
//RGBA[Index] = 0x0F0000FF;
}
for(; Index < BUFFERSIZE/4; Index ++) {
/*
* Green at the bottom half
* Alpha = 0xF0
* */
//RGBA[Index] = 0xF000FF00;
}
return Frame;
}
测试
应用程序完成后，我们可以创建调试配置并下载位文件并运行应用程序软件。

在监视器上应该能看到图像，但是我们也可以检查 ILA。这些显示 ILA 将显示从 MIPI CSI-2 IP 内核和 VDMA 接收的输入视频流。

AXIS 使用边带信号 Tuser 和 Tlast 传输视频帧。Tuser 表示新帧的开始，而 Tlast 表示一行的结束。

第三个 ILA 监控 AXI Stream to Video Output，使我们能够监控输出上的锁定信号。

当所有这些都运行时，将能够看到图像输出:

解决方案的最终资源利用率如下图所示，我们还有足够的空间来实现有趣的算法

源代码
https://github.com/ATaylorCEngFIET/Genesys_ZU_MIPI_PCAM