vNVFRUC_PG-09417-001_v03 | February2023
NVIDIA Optical Flow Engine-Assisted
Frame Rate Up Conversion
Programming Guide
NVIDIA Optical Flow Engine-Assisted Frame Rate Up
Conversion
vNVFRUC_PG-09417-001_v03|ii
Table of Contents
Chapter1.Introduction........................................................................................................ 1
1.1.Frame Rate Up Conversion......................................................................................................1
1.2.NVIDIA FRUC library.................................................................................................................2
Chapter2.How FRUC Library Works?................................................................................ 3
2.1.FRUC Library Usage Overview.................................................................................................3
2.2.Inside FRUC Library..................................................................................................................4
2.3.FRUC Library Components.......................................................................................................5
Chapter3.Programming Using FRUC APIs........................................................................ 6
3.1.Basic Programming Flow.........................................................................................................6
3.2.Background............................................................................................................................... 6
3.3.Example Of FRUC API Usage...................................................................................................7
3.4.Using FRUC APIs...................................................................................................................... 8
3.4.1.Creating FRUC Instance.................................................................................................... 8
3.4.2.Registering Resources.......................................................................................................9
3.4.3.Interpolating Intermediate Frame.....................................................................................9
3.4.4.Unregistering Resources................................................................................................. 11
3.4.5.Destroying FRUC Instance...............................................................................................12
3.4.6.Diagnostics........................................................................................................................12
Chapter4.Prerequisites.................................................................................................... 13
NVIDIA Optical Flow Engine-Assisted Frame Rate Up
Conversion
vNVFRUC_PG-09417-001_v03|1
Chapter1. Introduction
NVIDIA GPUs starting from Turing generation contain a hardware-based Optical Flow
Accelerator (NVOFA) that gives flow vectors map between the two frames. NVIDIA Optical Flow
SDK gives access to NVOFA via Optical Flow APIs.
NVIDIA Optical Flow SDK 4.0 and later SDKs include NVOFA assisted Frame Rate Up
Conversion (FRUC) library. The library exposes FRUC APIs that can be used for frame rate up
conversion of game or video.
This document provides information on how to use the FRUC APIs. It is expected that the
developer should have familiarity with Windows and/or Linux development environment.
1.1. Frame Rate Up Conversion
Frame Rate Up Conversion is a technique that generates higher frame rate video from lower
frame rate video by inserting interpolated frames into it. Such high frame rate video shows
smooth continuity of motion across frames that improves perceived visual quality of video.
Figure1. Frame Rate Up Conversion
Introduction
NVIDIA Optical Flow Engine-Assisted Frame Rate Up
Conversion
vNVFRUC_PG-09417-001_v03|2
1.2. NVIDIA FRUC library
NVIDIA FRUC library exposes FRUC APIs that take two consecutive frames and generate
an interpolated frame in between. These APIs can be used for frame rate up conversion of
gaming or video content.
The library internally uses NVOFA hardware engine and CUDA. As a result, the frame
interpolation using FRUC library is much faster compared to software-only methods.
The library supports ARGB and NV12 input surface formats. It can be directly integrated into a
DirectX game or a CUDA application.
The FRUC library works on Windows OS (Windows 10 and above) and Linux OS (distributions
Ubuntu 18 and above). It needs NVIDIA display driver version 511.65 or above on Windows and
NVIDIA display driver version 510.47.03 or above on Linux.
NVIDIA Optical Flow Engine-Assisted Frame Rate Up
Conversion
vNVFRUC_PG-09417-001_v03|3
Chapter2. How FRUC Library Works?
2.1. FRUC Library Usage Overview
Here is a block diagram showing how applications can use FRUC library for frame rate up
conversion.
Figure2. FRUC Library Software Stack
1. Application passes consecutive frames to FRUC library.
2. FRUC library uses frame from current call ( next frame ) and cached frame from previous
call ( previous frame ) to interpolate intermediate frame. The library first calls NVIDIA
Optical flow APIs to get forward and backward flow vector maps between the two frames
(previous and next). It then uses CUDA accelerated techniques to generate an interpolated
frame between the two frames.
