Building OpenCV with FastCV {#tutorial_building_fastcv}

| | | | -: | :- | | Compatibility | OpenCV >= 4.11.0 |

Enable OpenCV with FastCV for Qualcomm Chipsets

This document scope is to guide the Developers to enable OpenCV Acceleration with FastCV for the Qualcomm chipsets with ARM64 architecture. Entablement of OpenCV with FastCV back-end on non-Qualcomm chipsets or Linux platforms other than Qualcomm Linux is currently out of scope.

About FastCV

FastCV provides two main features to computer vision application developers:

A library of frequently used computer vision (CV) functions, optimized to run efficiently on a wide variety of Qualcomm’s Snapdragon devices.
A clean processor-agnostic hardware acceleration API, under which chipset vendors can hardware accelerate FastCV functions on Qualcomm’s Snapdragon hardware.

FastCV is released as a unified binary, a single binary containing two implementations of the algorithms:

Generic implementation runs on Arm® architecture and is referred to as FastCV for Arm architecture.
Implementation runs only on Qualcomm® Snapdragon™ chipsets and is called FastCV for Snapdragon.

FastCV library is Qualcomm proprietary and provides faster implementation of CV algorithms on various hardware as compared to other CV libraries.

OpenCV Acceleration with FastCV HAL and Extensions

OpenCV and FastCV integration is implemented in two ways:

FastCV-based HAL for basic computer vision and arithmetic algorithms acceleration.
FastCV module in opencv_contrib with custom algorithms and FastCV function wrappers that do not fit generic OpenCV interface or behaviour.

Supported Platforms

Android : Qualcomm Chipsets with the Android from Snapdragon 8 Gen 1 onwards(https://www.qualcomm.com/products/mobile/snapdragon/smartphones#product-list)
Linux : Qualcomm Linux Program related boards mentioned in Hardware

Compiling OpenCV with FastCV for Android

Follow Wiki page for OpenCV Compilation : https://github.com/opencv/opencv/wiki/Custom-OpenCV-Android-SDK-and-AAR-package-build

Once the OpenCV repository code is cloned into the workspace, please add -DWITH_FASTCV=ON flag to cmake vars as below to arm64 entry in opencv/platforms/android/default.config.py or create new one with the option to enable FastCV HAL and/or extenstions compilation:

  ABI("3", "arm64-v8a", None, 24, cmake_vars=dict(WITH_FASTCV='ON')),

Remaining steps can be followed as mentioned in the wiki page

Compiling OpenCV with FastCV for Qualcomm Linux

@note: Only Ubuntu 22.04 is supported as host platform for eSDK deployment.

Install eSDK by following Qualcomm® Linux Documentation
After installing the eSDK, set the ESDK_ROOT:
```
export ESDK_ROOT=<eSDK install location>
```
Add SDK tools and libraries to your environment:
```
source environment-setup-armv8-2a-qcom-linux
```

If you encounter the following message:

  Your environment is misconfigured, you probably need to 'unset LD_LIBRARY_PATH'
  but please check why this was set in the first place and that it's safe to unset.
  The SDK will not operate correctly in most cases when LD_LIBRARY_PATH is set.

just unset your host LD_LIBRARY_PATH environment variable: unset LD_LIBRARY_PATH.

Clone OpenCV Repositories:

Clone the OpenCV main and optionally opencv_contrib repositories into any directory (it does not need to be inside the SDK directory).

  git clone https://github.com/opencv/opencv.git
  git clone https://github.com/opencv/opencv_contrib.git

Build OpenCV

Create a build directory, navigate into it and build the project with CMake there:

  mkdir build
  cd build
  cmake -DCMAKE_SYSTEM_NAME=Linux -DCMAKE_SYSTEM_PROCESSOR=aarch64 -DWITH_FASTCV=ON -DBUILD_SHARED_LIBS=ON -DOPENCV_EXTRA_MODULES_PATH=../opencv_contrib/modules/fastcv/ ../opencv
  make -j$(nproc)

If the FastCV library is updated, please replace the old FastCV libraries located at:

  <ESDK_PATH>\qcom-wayland_sdk\tmp\sysroots\qcs6490-rb3gen2-vision-kit\usr\lib

with the latest FastCV libraries downloaded in:

  build\3rdparty\fastcv\libs

Validate

Push the OpenCV libraries, test binaries and test data on to the target. Execute the OpenCV conformance or performance tests. During runtime, If libwebp.so.7 lib is missing, find the lib in the below Path and push it on the target

  <ESDK_PATH>\qcom-wayland_sdk\tmp\sysroots\qcs6490-rb3gen2-vision-kit\usr\lib\libwebp.so.7

HAL and Extension list of APIs

FastCV based OpenCV HAL APIs list :

OpenCV module	OpenCV API	Underlying FastCV API for OpenCV acceleration
IMGPROC	medianBlur	fcvFilterMedian3x3u8_v3
	sobel	fcvFilterSobel3x3u8s16
		fcvFilterSobel5x5u8s16
		fcvFilterSobel7x7u8s16
	boxFilter	fcvBoxFilter3x3u8_v3
		fcvBoxFilter5x5u8_v2
		fcvBoxFilterNxNf32
	adaptiveThreshold	fcvAdaptiveThresholdGaussian3x3u8_v2
		fcvAdaptiveThresholdGaussian5x5u8_v2
		fcvAdaptiveThresholdMean3x3u8_v2
		fcvAdaptiveThresholdMean5x5u8_v2
	pyrDown	fcvPyramidCreateu8_v4
	cvtColor	fcvColorRGB888toYCrCbu8_v3
		fcvColorRGB888ToHSV888u8
	gaussianBlur	fcvFilterGaussian5x5u8_v3
		fcvFilterGaussian3x3u8_v4
	warpPerspective	fcvWarpPerspectiveu8_v5
	Canny	fcvFilterCannyu8

CORE	lut	fcvTableLookupu8
	norm	fcvHammingDistanceu8
	multiply	fcvElementMultiplyu8u16_v2
	transpose	fcvTransposeu8_v2
		fcvTransposeu16_v2
		fcvTransposef32_v2
	meanStdDev	fcvImageIntensityStats_v2
	flip	fcvFlipu8
		fcvFlipu16
		fcvFlipRGB888u8
	rotate	fcvRotateImageu8
		fcvRotateImageInterleavedu8
	multiply	fcvElementMultiplyu8
		fcvElementMultiplys16
		fcvElementMultiplyf32
	addWeighted	fcvAddWeightedu8_v2
	subtract	fcvImageDiffu8f32_v2
	SVD & solve	fcvSVDf32_v2
	gemm	fcvMatrixMultiplyf32_v2
		fcvMultiplyScalarf32
		fcvAddf32_v2

FastCV based OpenCV Extensions APIs list :

These OpenCV extension APIs are implemented under the cv::fastcv namespace.

OpenCV Extension APIs	Underlying FastCV API for OpenCV acceleration
matmuls8s32	fcvMatrixMultiplys8s32
clusterEuclidean	fcvClusterEuclideanu8
FAST10	fcvCornerFast10InMaskScoreu8
	fcvCornerFast10InMasku8
	fcvCornerFast10Scoreu8
	fcvCornerFast10u8
FFT	fcvFFTu8
IFFT	fcvIFFTf32
fillConvexPoly	fcvFillConvexPolyu8
houghLines	fcvHoughLineu8
moments	fcvImageMomentsu8
	fcvImageMomentss32
	fcvImageMomentsf32
runMSER	fcvMserInit
	fcvMserNN8Init
	fcvMserExtu8_v3
	fcvMserExtNN8u8
	fcvMserNN8u8
	fcvMserRelease
remap	fcvRemapu8_v2
remapRGBA	fcvRemapRGBA8888BLu8
	fcvRemapRGBA8888NNu8
resizeDown	fcvScaleDownBy2u8_v2
	fcvScaleDownBy4u8_v2
	fcvScaleDownMNInterleaveu8
	fcvScaleDownMNu8
meanShift	fcvMeanShiftu8
	fcvMeanShifts32
	fcvMeanShiftf32
bilateralRecursive	fcvBilateralFilterRecursiveu8
thresholdRange	fcvFilterThresholdRangeu8_v2
bilateralFilter	fcvBilateralFilter5x5u8_v3
	fcvBilateralFilter7x7u8_v3
	fcvBilateralFilter9x9u8_v3
calcHist	fcvImageIntensityHistogram
gaussianBlur	fcvFilterGaussian3x3u8_v4
	fcvFilterGaussian5x5u8_v3
	fcvFilterGaussian5x5s16_v3
	fcvFilterGaussian5x5s32_v3
	fcvFilterGaussian11x11u8_v2
filter2D	fcvFilterCorrNxNu8
	fcvFilterCorrNxNu8s16
	fcvFilterCorrNxNu8f32
sepFilter2D	fcvFilterCorrSepMxNu8
	fcvFilterCorrSep9x9s16_v2
	fcvFilterCorrSep11x11s16_v2
	fcvFilterCorrSep13x13s16_v2
	fcvFilterCorrSep15x15s16_v2
	fcvFilterCorrSep17x17s16_v2
	fcvFilterCorrSepNxNs16
sobel3x3u8	fcvImageGradientSobelPlanars8_v2
sobel3x3u8	fcvImageGradientSobelPlanars16_v2
sobel3x3u8	fcvImageGradientSobelPlanars16_v3
sobel3x3u8	fcvImageGradientSobelPlanarf32_v2
sobel3x3u8	fcvImageGradientSobelPlanarf32_v3
sobel	fcvFilterSobel3x3u8_v2
	fcvFilterSobel3x3u8s16
	fcvFilterSobel5x5u8s16
	fcvFilterSobel7x7u8s16
DCT	fcvDCTu8
iDCT	fcvIDCTs16
sobelPyramid	fcvPyramidAllocate
	fcvPyramidAllocate_v2
	fcvPyramidAllocate_v3
	fcvPyramidSobelGradientCreatei8
	fcvPyramidSobelGradientCreatei16
	fcvPyramidSobelGradientCreatef32
	fcvPyramidDelete
	fcvPyramidDelete_v2
	fcvPyramidCreatef32_v2
	fcvPyramidCreateu8_v4
trackOpticalFlowLK	fcvTrackLKOpticalFlowu8_v3
	fcvTrackLKOpticalFlowu8
warpPerspective2Plane	fcv2PlaneWarpPerspectiveu8
warpPerspective	fcvWarpPerspectiveu8_v5
arithmetic_op	fcvAddu8
	fcvAdds16_v2
	fcvAddf32
	fcvSubtractu8
	fcvSubtracts16
integrateYUV	fcvIntegrateImageYCbCr420PseudoPlanaru8
normalizeLocalBox	fcvNormalizeLocalBoxu8
	fcvNormalizeLocalBoxf32
merge	fcvChannelCombine2Planesu8
	fcvChannelCombine3Planesu8
	fcvChannelCombine4Planesu8
split	fcvDeinterleaveu8
	fcvChannelExtractu8
warpAffine	fcvTransformAffineu8_v2
	fcvTransformAffineClippedu8_v3
	fcv3ChannelTransformAffineClippedBCu8

FastCV QDSP based OpenCV Extension APIs list : These OpenCV extension APIs are implemented under the cv::fastcv::dsp namespace. This namespace provides optimized implementations that leverage QDSP (Qualcomm's Digital Signal Processor) acceleration using FastCV's Q-suffixed APIs. These functions require DSP initialization (fcvQ6Init).

OpenCV Extension APIs	Underlying FastCV API for OpenCV acceleration
filter2D	fcvFilterCorr3x3s8_v2Q
	fcvFilterCorrNxNu8Q
	fcvFilterCorrNxNu8s16Q
	fcvFilterCorrNxNu8f32Q
FFT	fcvFFTu8Q
IFFT	fcvIFFTf32Q
fcvdspinit	fcvQ6Init
fcvdspdeinit	fcvQ6DeInit
Canny	fcvFilterCannyu8Q
sumOfAbsoluteDiffs	fcvSumOfAbsoluteDiffs8x8u8_v2Q
thresholdOtsu	fcvFilterThresholdOtsuu8Q

How to Use FastCV QDSP based OpenCV Extension APIs

This section outlines the essential steps required to use OpenCV Extension APIs that are accelerated using FastCV on QDSP(Qualcomm's Digital Signal Processor).

Initialize QDSP:
- Call cv::fastcv::dsp::fcvdspinit() to initialize the QDSP.
Allocate memory using Qualcomm's memory allocator for all buffers that are being fed to the OpenCV extension API.:
- Use cv::fastcv::getQcAllocator() to assign the allocator to the buffers.
- Example: cv::Mat src; src.allocator = cv::fastcv::getQcAllocator(); // Set Qualcomm's memory allocator \ After setting Qualcomm's memory allocator, any buffer created using methods like src.create(...), cv::imread(...) etc., will have its memory allocated using Qualcomm's memory allocator.
Call the OpenCV extension API from 'cv::fastcv::dsp':
- Example: cv::fastcv::dsp::thresholdOtsu(src, dst, binaryType); where 'src' and 'dst' are 'cv::Mat' objects with the Qualcomm's memory allocator, and 'binaryType' is a boolean indicating the thresholding mode.
Deinitialize QDSP:
- Call cv::fastcv::dsp::fcvdspdeinit() to deinitialize the QDSP.

Reference Example: Refer to a working test case using the OpenCV Extension APIs in the opencv_contrib repository:opencv_contrib/modules/fastcv/test/test_thresh_dsp.cpp

building_fastcv.markdown 18 KB パーマリンク 履歴 Raw