Real-Time Video Abstraction:
Selected Source Code
1 Introduction
This file contains the core fragment code needed to create a GPU implementation of the image abstraction
filters described in our paper, Real-Time Video Abstraction [5].
The source provided uses the NVIDIA Cg language [2]. In most of the code, we assume that pos refers
to the image space position of the current pixel, and that tex refer s to the texture that we are reading from.
1
2 Blurring
What follows are two Cg-functions that implement a separated Gaus sian blur. Such blurs are used in our
framework during the difference of Gaussians edge emphasis step as well as in the image based warping step.
We define the following variables:
r ← ciel(3σ), inv_2sigma2 ←
1
2σ
2
, inv_sigma_root_2pi ←
1
σ
√
2π
.
Separated Blurring (part 1)
// Perform the x-blur pass
float gauss_x(in float2 pos : TEXCOORD0,
uniform samplerRECT tex,
uniform int r,
uniform float inv_2sigma2,
uniform float inv_sigma_root_2pi
) : COLOR {
int i;
float rval=texRECT(tex,pos);
for(i=1;i<r;i++) {
rval += exp(-i
*
i
*
inv_2sigma2)
*
texRECT(tex,pos+float2( i,0));
rval += exp(-i
*
i
*
inv_2sigma2)
*
texRECT(tex,pos+float2(-i,0));
}
return inv_sigma_root_2pi
*
rval;
}
Separated Blurring (part 2)
// Perform the y-blur pass
float gauss_y(in float2 pos : TEXCOORD0,
uniform samplerRECT tex,
uniform int r,
uniform float inv_2sigma2,
uniform float inv_sigma_root_2pi
) : COLOR {
int i;
float rval=texRECT(tex,pos);
for(i=1;i<r;i++) {
rval += exp(-i
*
i
*
inv_2sigma2)
*
texRECT(tex,pos+float2(0, i));
rval += exp(-i
*
i
*
inv_2sigma2)
*
texRECT(tex,pos+float2(0,-i));
}
return inv_sigma_root_2pi
*
rval;
}
Profiling tests done on a GeForce 6800 GT suggested that manually unrolling the loops involved in a
Cg implementa tion of a Gaussian blur, and caching the multiplication constants, is 10-20% faster than
implementing those same blurs using a for lo op, as above .
In our optimized implementation, fo r any given σ , we prog rammatically generate fragment code that is
functionally equivalent to the above, in which the loops have been unrolled, and the multiplicative constants
hard coded into the source file.
Precalculating the multiplication constants also gives us an opportunity to normalize them (i.e., we can
scale the multiplication constants such that they sum to exactly 1).
2
3 A Separated Approximation to the Bilateral Filter
From an implementation standpoint; a simple way to understand the separated approximation to the bilateral
is as a separated Gaussian blur, in which the normal blur coefficients are scaled by a range weig hting term,
e
−
||f(x)−f (ˆx)||
2σ
2
r
.
The following shows how we modify the x-gaussian blur pass above to create the x-pas s of a separated
approximation to a bilateral filter, as discussed in Pham and Vliet [4 ]. We introduce ra nge weighting
variables, which follow the same conventions as our sigma-variables.
inv_2vsigma2 ←
1
2σ
2
r
, inv_vsigma_root_2pi ←
1
σ
r
√
2π
.
Separated Approximation to the Bilateral Filter, X-pass
int i;
float3 current=texRECT(tex,pos);
// sum will store the numerator of the filter equation
// (see Eq. (1) in our paper)
float3 sum = current;
// while denom will store the denominator
float denom = 1;
for(i=1;i<r;i++) {
float3 next = texRECT(tex,pos+float2( i,0));
float3 prev = texRECT(tex,pos+float2(-i,0));
float3 next_displace = next - current;
float3 prev_displace = prev - current;
// We use a dot product to generate squared displacement lengths
float dist_to_next2 = dot(next_displace,next_displace);
float dist_to_prev2 = dot(prev_displace,prev_displace);
// Here we calculate the range weights
float next_weight =exp(inv_2vsigma2
*
(-dist_to_next2));
float prev_weight =exp(inv_2vsigma2
*
(-dist_to_prev2));
// And here we use them to scale the spatial weights
float m_next=exp(-i
*
i
*
inv_2sigma2)
*
next_weight;
float m_prev=exp(-i
*
i
*
inv_2sigma2)
*
prev_weight;
// Update the denominator and numerator of the filter equation
denom += m_next;
denom += m_prev;
sum += m_next
*
next;
sum += m_prev
*
prev;
}
return sum/denom;
Like the blurring code, the separated approximate bilateral filter benefits from loop unrolling and the
use of precalculated co nstants.
3
