Welcome to Scribd, the world's digital library. Read, publish, and share books and documents. See more
Download
Standard view
Full view
of .
Look up keyword
Like this
4Activity
0 of .
Results for:
No results containing your search query
P. 1
Super-Resolution Based on Open Source Computer Vision Library EmguCV

Super-Resolution Based on Open Source Computer Vision Library EmguCV

Ratings: (0)|Views: 3,326|Likes:
Published by JeeShen Lee
Super-Resolution implemented using EmguCV.
Super-Resolution implemented using EmguCV.

More info:

Categories:Types, Research
Published by: JeeShen Lee on Jun 06, 2013
Copyright:Attribution Non-commercial

Availability:

Read on Scribd mobile: iPhone, iPad and Android.
download as PDF, TXT or read online from Scribd
See more
See less

12/02/2014

pdf

text

original

 
Super-Resolution Based on Open Source Computer Vision Library EmguCV
Lee Jee Shen
Faculty of Engineering and TechnologyMultimedia UniversityMalacca, Malaysiame@jeeshenlee.com
Authors Name/s per 2nd Affiliation
(Author)
 
Dept. name of organization
(Line 1 of Affiliation - optional)
  Name of organization - acronyms acceptable
(line 2)
 City, Country
(line 3)
 name@xyz.com
 – 
optional
(line 4)
 
Abstract 
 — 
In this paper, we present a simple implementation of Super-Resolution technique using EmguCV computer visionlibrary. Instead of static images, we use real-time video inputsfrom web camera as the input. Each Super-Resolution imageresult are processed based on multiple frames of the video input.The Super-Resolution technique applied our application isimplemented with the following sequence
 – 
image acquisition,image registration, sub-pixel image shift calculation, imagetransformation and lastly image interpolation.
 Index Terms 
 — 
Super-Resolution, EmguCV, real-time videoinput image processing
I.
 
I
 NTRODUCTION
 Super-Resolution is an image processing technique thattakes multiple shifted images of a same scene and generate ahigher resolution image copy of the scene by considering lostimage information that are captured at the other frames of thesame scene due to vibration or image shift. The main concept behind Super-Resolution is that CCD camera sensors are finiteand discrete while the light signal is continuous. In one imageframe taken by CCD camera, there are many lost informationof the scene that does not get picked up by the camera sensor grid because some continuous light signal will not hit the CCDsensor grid directly. Figure 1 illustrate the scenario when a fullhit of light signal on CCD camera sensor grid while the secondimage (Fig.2) illustrate the scenario of none full hit(information lost) when the light hit in the might of the sensor grid.EmguCV is an open source computer vision library writtenon .NET platform. It
s a translated copy of OpenCV with someadditional feature specifically written for .NET programmer.EmguCV act as a wrapper of OpenCV to access memory leveldata which was considered as (unsafe) programming in the.NET environment. The library is written using C#.NET programming languages and it can be compiled into Window,Linux, Max OS, Android compatible with the Mono compiler.Mono Mono is a compiler written to compile C# languagestandard into the target platform. On top of EmguCV, we areusing another open source computer vision library AForgespecifically to take web camera input.
Fig. 1. CCD sensor are recording the intensity of the light based on theamount of intensity hit the sensor grid.Fig. 2. Extra information (the tip feature of the object) are better recorded.
II.
 
S
UPER 
-
RESOLUTION TECHNIQUE
 There are many Super-Resolution techniques proposed butgenerally it involves the following steps
 – 
image acquisition,image registration, sub-pixel shift estimation, and interpolation.In our application, we started by first acquiring the first five
 
image frames from web camera and then feed them into theSuper-Resolution image processing function that weimplemented to get the first Super-Resolution of our videostream. From then onward, we repeatedly update the Super-Resolution frame based on the latest video frames acquiredfrom the web camera.
Fig. 3. Three image video frames (on the left) are taken in this example onthe same scene. Based on the feature, sub-pixel shift of the scene aredetected on Frame 2 and Frame 3.Fig. 4. Sub-pixel shift are estimated and corrected before merging into thefinal result to produce better estimation of the real scene (Super-Resolution).
III.
 
V
IDEO FRAME ACQUISITION
 Image acquisition from the web camera are done using theAForge library.
VideoCaptureDeviceForm
class are used todetect and selected the web camera. After the selection,
 IVideoSource
passed and initialised. The
 NewFrame
event of the
 IVideoSource
are subscribed in the application to grab thelatest video frame from web camera and further triggering theSuper-Resolution processing. On top of this, we added afeature to define Region of Interest (ROI) to select specificregion of an image input for processing. Region of Interestedare registered using
System.Windows.Rectangle
class with thehelp of 
 MouseDown
and
 MouseUp
event.IV.
 
I
MAGE REGISTRATION
 Image registration is the most important part of Super-Resolution because it
s responsible for the feature detectionand also the sub-pixel calculation that determine the quality for the output. To make out Super-Resolution implementationreusable in other application, we create a separate project andwrote our 
SuperResolutionFilter 
class. In the class, we definedthe following object properties:TABLE I.
CLASS PROPERTY
 
Property Name Description
ScaleFactor Storing the scaling factor of the target output.FramesFusionAccuracyThresholdMinimum accuracy threshold that the shiftresult confidence must comply beforeconsidering as a valid estimationInputFramesList of input frames to store the latest framesfrom web camera.SuperResolutionFrameStoring the latest Super-Resolution imageresult.Roi Storing the Region of Interest of the image.ShiftResultStoring the X and Y shifted pixel in float datatype.
In order to find the sub-pixel of images with reference tothe first image, we first find a feature on an image to track. Weare using
GoodFeaturesToTrack 
method of EmguCV to detectfeatures on the video inputs. Before the images are pass to
GoodFeaturesToTrack 
it was convert into the right imageformat required by EmguCV which is
 Image<Grey, byte>
. Toget better result, the resulted features detected are further 
 
 passed to
 FindCornerSubPix
method of EmguCV to get sub- pixel accurate corner of the image.Based on the features found, a sub-pixel shift of importantfeature are calculated using
OpticalFlow.PyrLK 
function.
OpticalFlow.PyrLK 
calculates the optical flow for a sparsefeatures set (detected in the earlier steps) using iterative Lucas-Kanade method in pyramids. To produce better result, weadded some additional code to calculate the confidence level of each calculated shift. This confidence level value will be usedlater in the merging steps to filtered inaccurate sub-pixel shiftestimation.Based on the calculated sub-pixel shift results andconfidence level, the images are transforms using
WarpPerspective
to correct the sub-pixel shift based on theestimation. Lastly, the images list are merge using
 RunningAvg 
 of EmguCV to produce the Super-Resolution image. Theapplication then repeat the same process describe above on allthe incoming video frames except this time using the latestSuper-Resolution image calculated in the last iteration asreference to produce better result.V.
 
E
XPERIMENTAL RESULTS
 To test the implemented Super-Resolution techniqueimplemented using EmguCV, we run our application withmany test conditions. Below are sample of the results recordedduring the test.
Fig. 5. The implemented Super-Resolution application using Visual Studio2013 and EmguCV.Fig. 6. On the left is the video stream from web camera and on the rightis the result stream generated live.Fig. 7. Improvement can be seen on the right-hand generated Super-Resolution image as compare to the original image taken from webcamera (on the right) when both images are zoom in for inspection.

Activity (4)

You've already reviewed this. Edit your review.
1 thousand reads
1 hundred reads

You're Reading a Free Preview

Download
scribd
/*********** DO NOT ALTER ANYTHING BELOW THIS LINE ! ************/ var s_code=s.t();if(s_code)document.write(s_code)//-->