Developing_Multimedia_Applications_with_NDK.pdf

Developing Multimedia

Applications with NDK

In this chapter, we will cover the following recipes:

f Porting the ffmpeg library to Android with NDK

f Using the ffmpeg library to get media info

f Decoding and displaying the video frame

f Separating decoding and drawing with two threads

f Seeking to playback and grabbing the frames

f Optimizing the performance of multimedia apps

potential to achieve better performance. Secondly, NDK allows us to detect CPU features and

you can take advantage of the CPU features offered by a particular device. For example, NEON

is a general-purpose Single Instruction Multiple Data (SIMD) engine available on many ARM

CPUs. It can speed up some media processing tasks (for example, color conversion) several

times. You can develop your apps in such a way that if NEON is available, it will use the NEON-

specic code. Thirdly, multimedia apps usually require manipulation of bits and bytes, moving

chunks of memory around, and so on. Native languages such as C/C++ are better at those

tasks than Java. Last but not least, Android only offers support for a few video/audio codecs.

There are a lot of open source codec libraries available and most of them are in native

languages. You can port them to Android using NDK to enable support for additional codecs.

a video le and save the frames being displayed to pictures. It is based on the well-known

ffmpeg library. We will rst port the ffmpeg library to Android with NDK. After that, you will

introduce the basics of reading a video le and initializing the codec context. The next step

is to decode, scale, and display the video frame. We will see how to do this with the Android

SurfaceView class, bitmap, and ffmpeg. We will also illustrate dividing the decoding,

scale, and display tasks into two threads, performing random seek at playback, and

saving the video frame to pictures. The last recipe will discuss the techniques to

optimize the performance.

The frame grabber app uses the ActionBarSherlock library for the GUI. You will link to

the library at all your sample projects in this chapter. The ActionBarSherlock library is

Getting ready

This recipe uses many of the techniques covered Chapter 8, Porting and Using Existing

Libraries with Android NDK. The following recipes should be reviewed rst in particular:

f Porting a library with its existing build system

f Using a library in multiple projects with import-module

How to do it...

The following steps describe how to build a ffmpeg library for Android with NDK:

1. Go to http://ffmpeg.org/download.html, and download the ffmpeg source

code. In this example, you used ffmpeg 1.0.1 release. Extract the downloaded archive

to the sources folder of our Android NDK directory. The extracted les will be put into

script to point to the correct location in our computer.

#!/bin/bash

NDK=/home/roman10/Desktop/android/android-ndk-r8b

SYSROOT=$NDK/platforms/android-8/arch-arm/

TOOLCHAIN=$NDK/toolchains/arm-linux-androideabi-4.6/prebuilt/

linux-x86

function build_one

{

./configure \

--prefix=$PREFIX \

--disable-ffserver \

--disable-avdevice \

--disable-doc \

--disable-symver \

--cross-prefix=$TOOLCHAIN/bin/arm-linux-androideabi- \

--target-os=linux \

--arch=arm \

--enable-cross-compile \

--sysroot=$SYSROOT \

--extra-cflags="-Os -fpic $ADDI_CFLAGS" \

CPU=arm

PREFIX=$(pwd)/android/$CPU

ADDI_CFLAGS="-marm"

build_one

$ chmod +x build_android.sh

$ ./build_android.sh

The build will take quite a while to nish. After it is done, you can nd the library

headers under the sources/ffmpeg-1.0.1/android/arm/include directory,

and library les under the sources/ffmpeg-1.0.1/android/arm/lib directory.

4. Add an Android.mk le under the sources/ffmpeg-1.0.1/android/arm folder:

include $(PREBUILT_STATIC_LIBRARY)

#static version of libavformat

include $(CLEAR_VARS)

LOCAL_MODULE:= libavformat_static

LOCAL_SRC_FILES:= lib/libavformat.a

LOCAL_EXPORT_C_INCLUDES := $(LOCAL_PATH)/include

include $(PREBUILT_STATIC_LIBRARY)

#static version of libswscale

include $(CLEAR_VARS)

LOCAL_MODULE:= libswscale_static

LOCAL_EXPORT_C_INCLUDES := $(LOCAL_PATH)/include

LOCAL_EXPORT_C_INCLUDES := $(LOCAL_PATH)/include

include $(PREBUILT_STATIC_LIBRARY)

How it works...

The previous steps show how to build ffmpeg for Android and prepare it to be used by

Android applications.

The idea is to build the ffmpeg library and allow multiple projects to link to it as prebuilt

libraries with import-module. You put the ffmpeg library source code under the sources folder

of the Android NDK directory. As discussed in the Using a library in multiple projects with

import-module recipe in Chapter 8, Porting and Using the Existing Libraries with Android

NDK, the sources directory is appended to NDK_MODULE_PATH by NDK build system.

libavcodec, libavlter, libavformat, libavutil, libswresample and libswscale.

After compilation is done, you provide an Android.mk le in step 4 for an Android application

project to easily import the ffmpeg libraries as static prebuilt library modules. You will see how

to use the ffmpeg libraries in subsequent recipes.

Using the ffmpeg library to get media info

Starting from this recipe, you will show how to use the ffmpeg libraries to develop multimedia

applications. This recipe will discuss how to obtain various information of a video le.

Getting ready

This recipe and subsequent recipes are based on a project template available from

the book's youbsite. Please download the source code for this book and locate the