将图像效果（Lumia Imaging SDK、Win2D、DirectX HLSL 像素着色器）和检测器（ZXing.Net ...

[](https://ci.appveyor.com/project/mmaitre314/videoeffect) [](https://www.nuget.org/packages/MMaitre.VideoEffects/) [](http://mmaitre314.github.io/2015/05/24/personal-pdb-symbol-server.html) Video Effects ============= Original|Antique + HorizontalFlip ----|---- | Apply image effects from the [Lumia Imaging SDK](https://msdn.microsoft.com/en-us/library/dn859593.aspx), [Win2D](https://github.com/Microsoft/Win2D), and [DirectX HLSL pixel shaders](http://msdn.microsoft.com/en-us/library/bb509635(v=VS.85).aspx) to videos in [Universal Store Apps](http://msdn.microsoft.com/en-us/library/windows/apps/dn609832.aspx) for Windows Phone 8.1 and Windows 8.1. Effects can be applied via [MediaTranscoder](http://msdn.microsoft.com/en-us/library/windows/apps/windows.media.transcoding.mediatranscoder.aspx), [MediaComposition](http://msdn.microsoft.com/en-us/library/windows/apps/xaml/windows.media.editing.mediacomposition.aspx), [MediaCapture](http://msdn.microsoft.com/en-us/library/windows/apps/xaml/windows.media.capture.mediacapture.aspx), or [MediaElement](http://msdn.microsoft.com/en-us/library/windows/apps/xaml/windows.ui.xaml.controls.mediaelement.aspx). Binaries are available via [NuGet](https://www.nuget.org/packages/MMaitre.VideoEffects/). Note for Win10+: the OS provides built-in support for video effects via [IBasicVideoEffect](https://msdn.microsoft.com/en-us/library/windows/apps/windows.media.effects.ibasicvideoeffect.aspx), which makes the VideoEffect lib obsolete. See [Win2D Example Gallery Effects](https://github.com/Microsoft/Win2D/tree/31bf8ac397e272bdadfdac1493ae3b52680d570a/samples/ExampleGallery/Effects) for code samples. Lumia Imaging SDK effects ------------------------- A video effect definition is created from a chain of image effects and passed to a video-processing class like MediaTranscoder: ```c# var definition = new LumiaEffectDefinition(() => { return new IFilter[] { new AntiqueFilter(), new FlipFilter(FlipMode.Horizontal) }; }); var transcoder = new MediaTranscoder(); transcoder.AddVideoEffect(definition.ActivatableClassId, true, definition.Properties); ``` ### Square videos Image effects changing the image resolution -- cropping for instance -- are also supported. In that case the resolutions of the input and output of the effect need to be specified explicitly. For instance, the following code snippet creates a square video: ```c# // Select the largest centered square area in the input video var encodingProfile = await TranscodingProfile.CreateFromFileAsync(file); uint inputWidth = encodingProfile.Video.Width; uint inputHeight = encodingProfile.Video.Height; uint outputLength = Math.Min(inputWidth, inputHeight); Rect cropArea = new Rect( (float)((inputWidth - outputLength) / 2), (float)((inputHeight - outputLength) / 2), (float)outputLength, (float)outputLength ); var definition = new LumiaEffectDefinition(() => { return new IFilter[] { new CropFilter(cropArea) }; }); definition.InputWidth = inputWidth; definition.InputHeight = inputHeight; definition.OutputWidth = outputLength; definition.OutputHeight = outputLength; ``` Note: in Windows Phone 8.1 a bug in MediaComposition prevents the width/height information to be properly passed to the effect. ### Overlays BlendFilter can overlay an image on top of a video: ```c# var file = await StorageFile.GetFileFromApplicationUriAsync(new Uri("ms-appx:///Assets/traffic.png")); var foreground = new StorageFileImageSource(file); var definition = new LumiaEffectDefinition(() => { var filter = new BlendFilter(foreground); filter.TargetOutputOption = OutputOption.PreserveAspectRatio; filter.TargetArea = new Rect(0, 0, .4, .4); return new IFilter[] { filter }; }); ``` ### Animations The LumiaEffectDefinition() constructor is overloaded to support effects whose properties vary based on time. This requires creating a class implementing the IAnimatedFilterChain interface, with a 'Filters' property returning the current effect chain and an 'UpdateTime()' method receiving the current time. ```c# class AnimatedWarp : IAnimatedFilterChain { WarpFilter _filter = new WarpFilter(WarpEffect.Twister, 0); public IEnumerable<IFilter> Filters { get; private set; } public void UpdateTime(TimeSpan time) { _filter.Level = .5 * (Math.Sin(2 * Math.PI * time.TotalSeconds) + 1); // 1Hz oscillation between 0 and 1 } public AnimatedWarp() { Filters = new List<IFilter> { _filter }; } } var definition = new LumiaEffectDefinition(() => { return new AnimatedWarp(); }); ``` ### Bitmaps and pixel data For cases where IFilter is not flexible enough, another overload of the LumiaEffectDefinition() constructor supports effects which handle Bitmap objects directly. This requires implementing IBitmapVideoEffect, which has a single Process() method called with an input bitmap, an output bitmap, and the current time for each frame in the video. The bitmaps passed to the Process() call get destroyed when the call returns, so any async call in this method must be executed synchronously using '.AsTask().Wait()'. The following code snippet shows how to apply a watercolor effect to the video: ```c# class WatercolorEffect : IBitmapVideoEffect { public void Process(Bitmap input, Bitmap output, TimeSpan time) { var effect = new FilterEffect(); effect.Filters = new IFilter[]{ new WatercolorFilter() }; effect.Source = new BitmapImageSource(input); var renderer = new BitmapRenderer(effect, output); renderer.RenderAsync().AsTask().Wait(); // Async calls must run sync inside Process() } } var definition = new LumiaEffectDefinition(() => { return new WatercolorEffect(); }); ``` IBitmapVideoEffect also allows raw pixel-data processing. Pixel data is provided in Bgra888 format (32 bits per pixel). The content of the alpha channel is undefined and should not be used. For efficiency, a GetData() method extension is provided on IBuffer. It is enabled by adding a 'using VideoEffectExtensions;' statement. GetData() returns an 'unsafe' byte* pointing to the IBuffer data. This requires methods calling GetData() to be marked using the 'unsafe' keyword and to check the 'Allow unsafe code' checkbox in the project build properties. The following code snippet shows how to apply a blue filter by setting both the red and green channels to zero: ```c# using VideoEffectExtensions; class BlueEffect : IBitmapVideoEffect { public unsafe void Process(Bitmap input, Bitmap output, TimeSpan time) { uint width = (uint)input.Dimensions.Width; uint height = (uint)input.Dimensions.Height; uint inputPitch = input.Buffers[0].Pitch; byte* inputData = input.Buffers[0].Buffer.GetData(); uint outputPitch = output.Buffers[0].Pitch; byte* outputData = output.Buffers[0].Buffer.GetData(); for (uint i = 0; i < height; i++) { for (uint j = 0; j < width; j++) { outputData[i * outputPitch + 4 * j + 0] = inputData[i * inputPitch + 4 * j + 0]; // B outputData[i * outputPitch + 4 * j + 1] = 0; // G outputData[i * outputPitch + 4 * j + 2] = 0; // R } } } } ``` ### Realtime video analysis and QR code detection  `LumiaAnalyzerDefinition` allows running image analysis realtime on video streams. It sends video frames to the app as it receives them without delaying frames inside the video stre