MIPI-CSI Debugging for RK3576 Processor — Channel Analysis | forlinx embedded

MIPI-CSI is a common camera interface in embedded systems or mobile devices, enabling high-speed image data transfer. The latest OK3576-C development board from Forlinx Embedded offers a wealth of resource interfaces, including support for 5 x CSI-2 , meaning it can support up to five camera inputs simultaneously.

This article introduces the Camera path of RK3576 processor and how to configure different output formats of MIPI-CSI camera through OK3576-C development board.

01. RK3576 Camera Channels

If there is only one camera access, turn on only rkispx_vir0 . Please note:

vicap and isp do not have a direct correspondence;
The relationships between different vir0/vir1 channels are essentially the same hardware being multiplexed, with the same effect. When using multiple cameras, configure them using 0, 1, 2, etc., as much as possible.

The hardware channel diagram is as follows:

There is one dcphy interface and two dphy interfaces. The connection paths are as follows:

Single Camera (connected to one dphy):

sensor0->csi2_dphy0->mipi1_csi2->rkcif_mipi_lvds1(sditf)->rkisp_vir0

Dual Cameras (connected to two dphys)

sensor0->csi2_dphy0->mipi1_csi2->rkcif_mipi_lvds1(sditf)->rkisp_vir0
sensor1->csi2_dphy3->mipi3_csi2->rkcif_mipi_lvds3(sditf)->rkisp_vir1

Three Cameras (connected to dcphy and two dphys)

sensor0->csi2_dcphy0->mipi0_csi2->rkcif_mipi_lvds(sditf)->rkisp_vir0
sensor1->csi2_dphy0->mipi1_csi2->rkcif_mipi_lvds1(sditf)->rkisp_vir1
sensor2->csi2_dphy3->mipi3_csi2->rkcif_mipi_lvds3(sditf)->rkisp_vir2

Five Cameras (connected to dcphy and two split dphys)

sensor0->csi2_dcphy0->mipi0_csi2->rkcif_mipi_lvds(sditf)->rkisp_vir0
sensor1->csi2_dphy1->mipi1_csi2->rkcif_mipi_lvds1(sditf)->rkisp_vir1
sensor2->csi2_dphy2->mipi2_csi2->rkcif_mipi_lvds2(sditf)->rkisp_vir2
sensor3->csi2_dphy4->mipi3_csi2->rkcif_mipi_lvds3(sditf)->rkisp_vir3
sensor4->csi2_dphy5->mipi4_csi2->rkcif_mipi_lvds4(sditf)->rkisp_vir4

Camera Channel Connection Diagram:

(Note: For RGB data input, it is also necessary to connect to rkisp_virx)

02. Sensor Link Scenarios for Different Platforms

yuv422/rgb888 input

There are three common scenarios for yuv422/rgb888 input:

Cameras with Built-in isp or External isp. Enter yuv422 format;
HDMI to MIPI CSI Input. Common chips like rk628d/f,lt6911xxx, etc., typically convert yuv422, but rgb888 format is also possible;
Multiple ahd and serdes. A single mipi port can support up to four virtual channels. For these scenarios, an isp is not required, and the connection only needs to go to cif. Thus, the link is:

sensor->csi2_dphyX->mipiX_csi2->rkcif_mipi_lvdsX rkcif_mipi_lvdsx_sditf

The isp node can be disabled. The X varies depending on the platform and hardware connections.

The node used for capturing images is the first video node corresponding to rkcif_mipi_lvdsX. This can be viewed using media-ctl to check the topology. For example, on the OK3576-C development board with an OV5645 camera, it is mounted on the media1 node.

root@ok3576-buildroot:/# media-ctl -p -d /dev/media1
Media controller API version 6.1.57
driver          rkcif
model           rkcif-mipi-lvds1
serial
bus info        platform:rkcif-mipi-lvds1
hw revision     0x0
driver version  6.1.57
Device topology
- entity 1: stream_cif_mipi_id0 (1 pad, 11 links)      type Node subtype V4L flags 0      device node name /dev/video11  pad0: Sink          <- "rockchip-mipi-csi2":1 [ENABLED] <- "rockchip-mipi-csi2":2 [] <- "rockchip-mipi-csi2":3 [] <- "rockchip-mipi-csi2":4 [] <- "rockchip-mipi-csi2":5 [] <- "rockchip-mipi-csi2":6 [] <- "rockchip-mipi-csi2":7 [] <- "rockchip-mipi-csi2":8 [] <- "rockchip-mipi-csi2":9 [] <- "rockchip-mipi-csi2":10 [] <- "rockchip-mipi-csi2":11 [] - entity 5: stream_cif_mipi_id1 (1 pad, 11 links) type Node subtype V4L flags 0 device node name /dev/video12 pad0: Sink <- "rockchip-mipi-csi2":1 [] <- "rockchip-mipi-csi2":2 [ENABLED] <- "rockchip-mipi-csi2":3 [] <- "rockchip-mipi-csi2":4 [] <- "rockchip-mipi-csi2":5 [] <- "rockchip-mipi-csi2":6 [] <- "rockchip-mipi-csi2":7 [] <- "rockchip-mipi-csi2":8 [] <- "rockchip-mipi-csi2":9 [] <- "rockchip-mipi-csi2":10 [] <- "rockchip-mipi-csi2":11 [] - entity 9: stream_cif_mipi_id2 (1 pad, 11 links) type Node subtype V4L flags 0 device node name /dev/video13 pad0: Sink <- "rockchip-mipi-csi2":1 [] <- "rockchip-mipi-csi2":2 [] <- "rockchip-mipi-csi2":3 [ENABLED] <- "rockchip-mipi-csi2":4 [] <- "rockchip-mipi-csi2":5 [] <- "rockchip-mipi-csi2":6 [] <- "rockchip-mipi-csi2":7 [] <- "rockchip-mipi-csi2":8 [] <- "rockchip-mipi-csi2":9 [] <- "rockchip-mipi-csi2":10 [] <- "rockchip-mipi-csi2":11 [] - entity 13: stream_cif_mipi_id3 (1 pad, 11 links) type Node subtype V4L flags 0 device node name /dev/video14 pad0: Sink <- "rockchip-mipi-csi2":1 [] <- "rockchip-mipi-csi2":2 [] <- "rockchip-mipi-csi2":3 [] <- "rockchip-mipi-csi2":4 [ENABLED] <- "rockchip-mipi-csi2":5 [] <- "rockchip-mipi-csi2":6 [] <- "rockchip-mipi-csi2":7 [] <- "rockchip-mipi-csi2":8 [] <- "rockchip-mipi-csi2":9 [] <- "rockchip-mipi-csi2":10 [] <- "rockchip-mipi-csi2":11 [] - entity 17: rkcif_scale_ch0 (1 pad, 11 links) type Node subtype V4L flags 0 device node name /dev/video15 pad0: Sink <- "rockchip-mipi-csi2":1 [] <- "rockchip-mipi-csi2":2 [] <- "rockchip-mipi-csi2":3 [] <- "rockchip-mipi-csi2":4 [] <- "rockchip-mipi-csi2":5 [ENABLED] <- "rockchip-mipi-csi2":6 [] <- "rockchip-mipi-csi2":7 [] <- "rockchip-mipi-csi2":8 [] <- "rockchip-mipi-csi2":9 [] <- "rockchip-mipi-csi2":10 [] <- "rockchip-mipi-csi2":11 [] - entity 21: rkcif_scale_ch1 (1 pad, 11 links) type Node subtype V4L flags 0 device node name /dev/video16 pad0: Sink <- "rockchip-mipi-csi2":1 [] <- "rockchip-mipi-csi2":2 [] <- "rockchip-mipi-csi2":3 [] <- "rockchip-mipi-csi2":4 [] <- "rockchip-mipi-csi2":5 [] <- "rockchip-mipi-csi2":6 [ENABLED] <- "rockchip-mipi-csi2":7 [] <- "rockchip-mipi-csi2":8 [] <- "rockchip-mipi-csi2":9 [] <- "rockchip-mipi-csi2":10 [] <- "rockchip-mipi-csi2":11 [] - entity 25: rkcif_scale_ch2 (1 pad, 11 links) type Node subtype V4L flags 0 device node name /dev/video17 pad0: Sink <- "rockchip-mipi-csi2":1 [] <- "rockchip-mipi-csi2":2 [] <- "rockchip-mipi-csi2":3 [] <- "rockchip-mipi-csi2":4 [] <- "rockchip-mipi-csi2":5 [] <- "rockchip-mipi-csi2":6 [] <- "rockchip-mipi-csi2":7 [ENABLED] <- "rockchip-mipi-csi2":8 [] <- "rockchip-mipi-csi2":9 [] <- "rockchip-mipi-csi2":10 [] <- "rockchip-mipi-csi2":11 [] - entity 29: rkcif_scale_ch3 (1 pad, 11 links) type Node subtype V4L flags 0 device node name /dev/video18 pad0: Sink <- "rockchip-mipi-csi2":1 [] <- "rockchip-mipi-csi2":2 [] <- "rockchip-mipi-csi2":3 [] <- "rockchip-mipi-csi2":4 [] <- "rockchip-mipi-csi2":5 [] <- "rockchip-mipi-csi2":6 [] <- "rockchip-mipi-csi2":7 [] <- "rockchip-mipi-csi2":8 [ENABLED] <- "rockchip-mipi-csi2":9 [] <- "rockchip-mipi-csi2":10 [] <- "rockchip-mipi-csi2":11 [] - entity 33: rkcif_tools_id0 (1 pad, 11 links) type Node subtype V4L flags 0 device node name /dev/video19 pad0: Sink <- "rockchip-mipi-csi2":1 [] <- "rockchip-mipi-csi2":2 [] <- "rockchip-mipi-csi2":3 [] <- "rockchip-mipi-csi2":4 [] <- "rockchip-mipi-csi2":5 [] <- "rockchip-mipi-csi2":6 [] <- "rockchip-mipi-csi2":7 [] <- "rockchip-mipi-csi2":8 [] <- "rockchip-mipi-csi2":9 [ENABLED] <- "rockchip-mipi-csi2":10 [] <- "rockchip-mipi-csi2":11 [] - entity 37: rkcif_tools_id1 (1 pad, 11 links) type Node subtype V4L flags 0 device node name /dev/video20 pad0: Sink <- "rockchip-mipi-csi2":1 [] <- "rockchip-mipi-csi2":2 [] <- "rockchip-mipi-csi2":3 [] <- "rockchip-mipi-csi2":4 [] <- "rockchip-mipi-csi2":5 [] <- "rockchip-mipi-csi2":6 [] <- "rockchip-mipi-csi2":7 [] <- "rockchip-mipi-csi2":8 [] <- "rockchip-mipi-csi2":9 [] <- "rockchip-mipi-csi2":10 [ENABLED] <- "rockchip-mipi-csi2":11 [] - entity 41: rkcif_tools_id2 (1 pad, 11 links) type Node subtype V4L flags 0 device node name /dev/video21 pad0: Sink <- "rockchip-mipi-csi2":1 [] <- "rockchip-mipi-csi2":2 [] <- "rockchip-mipi-csi2":3 [] <- "rockchip-mipi-csi2":4 [] <- "rockchip-mipi-csi2":5 [] <- "rockchip-mipi-csi2":6 [] <- "rockchip-mipi-csi2":7 [] <- "rockchip-mipi-csi2":8 [] <- "rockchip-mipi-csi2":9 [] <- "rockchip-mipi-csi2":10 [] <- "rockchip-mipi-csi2":11 [ENABLED] - entity 45: rockchip-mipi-csi2 (12 pads, 122 links) type V4L2 subdev subtype Unknown flags 0 device node name /dev/v4l-subdev1 pad0: Sink [fmt:UYVY8_2X8/1920x1080 field:none colorspace:srgb crop.bounds:(0,0)/1920x1080 crop:(0,0)/1920x1080] <- "rockchip-csi2-dphy0":1 [ENABLED] pad1: Source -> "stream_cif_mipi_id0":0 [ENABLED]          -> "stream_cif_mipi_id1":0 []          -> "stream_cif_mipi_id2":0 []          -> "stream_cif_mipi_id3":0 []          -> "rkcif_scale_ch0":0 []          -> "rkcif_scale_ch1":0 []          -> "rkcif_scale_ch2":0 []          -> "rkcif_scale_ch3":0 []          -> "rkcif_tools_id0":0 []          -> "rkcif_tools_id1":0 []          -> "rkcif_tools_id2":0 []  pad2: Source          -> "stream_cif_mipi_id0":0 []          -> "stream_cif_mipi_id1":0 [ENABLED]          -> "stream_cif_mipi_id2":0 []          -> "stream_cif_mipi_id3":0 []          -> "rkcif_scale_ch0":0 []          -> "rkcif_scale_ch1":0 []          -> "rkcif_scale_ch2":0 []          -> "rkcif_scale_ch3":0 []          -> "rkcif_tools_id0":0 []          -> "rkcif_tools_id1":0 []          -> "rkcif_tools_id2":0 []  pad3: Source          -> "stream_cif_mipi_id0":0 []          -> "stream_cif_mipi_id1":0 []          -> "stream_cif_mipi_id2":0 [ENABLED]          -> "stream_cif_mipi_id3":0 []          -> "rkcif_scale_ch0":0 []          -> "rkcif_scale_ch1":0 []          -> "rkcif_scale_ch2":0 []          -> "rkcif_scale_ch3":0 []          -> "rkcif_tools_id0":0 []          -> "rkcif_tools_id1":0 []          -> "rkcif_tools_id2":0 []  pad4: Source          -> "stream_cif_mipi_id0":0 []          -> "stream_cif_mipi_id1":0 []          -> "stream_cif_mipi_id2":0 []          -> "stream_cif_mipi_id3":0 [ENABLED]          -> "rkcif_scale_ch0":0 []          -> "rkcif_scale_ch1":0 []          -> "rkcif_scale_ch2":0 []          -> "rkcif_scale_ch3":0 []          -> "rkcif_tools_id0":0 []          -> "rkcif_tools_id1":0 []          -> "rkcif_tools_id2":0 []  pad5: Source          -> "stream_cif_mipi_id0":0 []          -> "stream_cif_mipi_id1":0 []          -> "stream_cif_mipi_id2":0 []          -> "stream_cif_mipi_id3":0 []          -> "rkcif_scale_ch0":0 [ENABLED]          -> "rkcif_scale_ch1":0 []          -> "rkcif_scale_ch2":0 []          -> "rkcif_scale_ch3":0 []          -> "rkcif_tools_id0":0 []          -> "rkcif_tools_id1":0 []          -> "rkcif_tools_id2":0 []  pad6: Source          -> "stream_cif_mipi_id0":0 []          -> "stream_cif_mipi_id1":0 []          -> "stream_cif_mipi_id2":0 []          -> "stream_cif_mipi_id3":0 []          -> "rkcif_scale_ch0":0 []          -> "rkcif_scale_ch1":0 [ENABLED]          -> "rkcif_scale_ch2":0 []          -> "rkcif_scale_ch3":0 []          -> "rkcif_tools_id0":0 []          -> "rkcif_tools_id1":0 []          -> "rkcif_tools_id2":0 []  pad7: Source          -> "stream_cif_mipi_id0":0 []          -> "stream_cif_mipi_id1":0 []          -> "stream_cif_mipi_id2":0 []          -> "stream_cif_mipi_id3":0 []          -> "rkcif_scale_ch0":0 []          -> "rkcif_scale_ch1":0 []          -> "rkcif_scale_ch2":0 [ENABLED]          -> "rkcif_scale_ch3":0 []          -> "rkcif_tools_id0":0 []          -> "rkcif_tools_id1":0 []          -> "rkcif_tools_id2":0 []  pad8: Source          -> "stream_cif_mipi_id0":0 []          -> "stream_cif_mipi_id1":0 []          -> "stream_cif_mipi_id2":0 []          -> "stream_cif_mipi_id3":0 []          -> "rkcif_scale_ch0":0 []          -> "rkcif_scale_ch1":0 []          -> "rkcif_scale_ch2":0 []          -> "rkcif_scale_ch3":0 [ENABLED]          -> "rkcif_tools_id0":0 []          -> "rkcif_tools_id1":0 []          -> "rkcif_tools_id2":0 []  pad9: Source          -> "stream_cif_mipi_id0":0 []          -> "stream_cif_mipi_id1":0 []          -> "stream_cif_mipi_id2":0 []          -> "stream_cif_mipi_id3":0 []          -> "rkcif_scale_ch0":0 []          -> "rkcif_scale_ch1":0 []          -> "rkcif_scale_ch2":0 []          -> "rkcif_scale_ch3":0 []          -> "rkcif_tools_id0":0 [ENABLED]          -> "rkcif_tools_id1":0 []          -> "rkcif_tools_id2":0 []  pad10: Source          -> "stream_cif_mipi_id0":0 []          -> "stream_cif_mipi_id1":0 []          -> "stream_cif_mipi_id2":0 []          -> "stream_cif_mipi_id3":0 []          -> "rkcif_scale_ch0":0 []          -> "rkcif_scale_ch1":0 []          -> "rkcif_scale_ch2":0 []          -> "rkcif_scale_ch3":0 []          -> "rkcif_tools_id0":0 []          -> "rkcif_tools_id1":0 [ENABLED]          -> "rkcif_tools_id2":0 []  pad11: Source          -> "stream_cif_mipi_id0":0 []          -> "stream_cif_mipi_id1":0 []          -> "stream_cif_mipi_id2":0 []          -> "stream_cif_mipi_id3":0 []          -> "rkcif_scale_ch0":0 []          -> "rkcif_scale_ch1":0 []          -> "rkcif_scale_ch2":0 []          -> "rkcif_scale_ch3":0 []          -> "rkcif_tools_id0":0 []          -> "rkcif_tools_id1":0 []          -> "rkcif_tools_id2":0 [ENABLED]
- entity 58: rockchip-csi2-dphy0 (2 pads, 2 links)      type V4L2 subdev subtype Unknown flags 0      device node name /dev/v4l-subdev2  pad0: Sink          [fmt:UYVY8_2X8/1920x1080@10000/300000 field:none colorspace:srgb           crop:(0,0)/1920x1080]          <- "m01_f_ov5645 3-003c":0 [ENABLED] pad1: Source -> "rockchip-mipi-csi2":0 [ENABLED]
- entity 63: m01_f_ov5645 3-003c (1 pad, 1 link)      type V4L2 subdev subtype Sensor flags 0      device node name /dev/v4l-subdev3  pad0: Source          [fmt:UYVY8_2X8/1920x1080@10000/300000 field:none colorspace:srgb           crop:(0,0)/1920x1080]          -> "rockchip-csi2-dphy0":0 [ENABLED]

If it is a multi-channel input, it corresponds to the first four, that is, video11-video14.

RAW format input

Starting from RK3588, that is, from isp3.0, the isp of Rockchip microprocessor no longer has the acquisition function, but only does image processing, so the whole path is:

sensor->csi2_dphyX->mipiX_csi2->rkcif_mipi_lvdsX rkcif_mipi_lvdsX_sditf->rkispx_virX

It needs to be explained here:

Without running aiq (Automatic Image Quality), rkcif_mipi_lvdsX can directly capture RAW images;
If only the isp node vir0, i.e., rkispx_vir0, is enabled and the subsequent nodes rkispx_vir1/vir2, etc., are not enabled, this is a direct pass-through configuration. Even without running aiq, nv12 images can still be obtained, but these images are unprocessed and generally appear in a light green color;
Enabling other vir nodes indicates that the isp needs to be time-multiplexed, which requires aiq to be enabled in order to capture images (aiq will handle time-multiplexing of the isp). Therefore, if only one camera is used, it is advisable to disable other isp nodes.

03. Summary

When configuring the camera, first determine the camera’s channel path and the format of the camera output. If it is RAW data, it must go through rkisp. If it is yuv422/rgb888, it only needs to be configured to rkcif_mipi_lvdsx.

This article introduced the camera channels of the RK3576 processor and explained how to configure different output formats for MIPI-CSI cameras. Future articles will continue to cover camera parameter configuration and device tree setup for the RK3576 processor.

MIPI-CSI Debugging for RK3576 Processor — Channel Analysis