You are on page 1of 39

Page |1

Camera Document
By Kristin Tanaka & Carlos Terrazas Table of Contents
Camera Document .................................................................................................................................................................. 1 Executive Summary ................................................................................................................................................................. 3 Project and System Level Requirements ................................................................................................................................ 3 System Level Task Descriptions .............................................................................................................................................. 3 Cost ......................................................................................................................................................................................... 4 Technical Divisions .................................................................................................................................................................. 5 Color Tracking and Edge Detection Track ............................................................................................................................... 5 Read articles on image processing .......................................................................................................................................... 5 Research Processing as a language and download environment to PC.................................................................................. 6 How to Download and Install Processing............................................................................................................................ 7 Example of Processing Development Environment ............................................................................................................ 9 Proceed with using examples and/or modifying code........................................................................................................ 9 Edge Detection example code ........................................................................................................................................ 9 Edge Detection example shows edges of leaves of jpeg file: ....................................................................................... 11 Simple color tracking example code: ............................................................................................................................ 11 Simple color tracking example shows tracking of blue color in center using webcam: ............................................... 12 Blob Detection example code ....................................................................................................................................... 13 Blob Detection example shows the detection of darker areas of image using webcam.............................................. 16 Edge Detection Background .............................................................................................................................................. 16 Types of edge detection.................................................................................................................................................... 17 Canny ............................................................................................................................................................................ 17 Prewitt........................................................................................................................................................................... 17 Sobel.............................................................................................................................................................................. 17 Laplacian ....................................................................................................................................................................... 17 Edge Detection Simulation in Matlab using webcam (Prewitt Method): ..................................................................... 17 Edge Detection example shows an image taken from webcam: .................................................................................. 18

Page |2 Image Capture Track ............................................................................................................................................................. 19 SD card breakout board .................................................................................................................................................... 20 File Allocation Table (FAT)................................................................................................................................................. 21 Connecting Camera to Arduino (shown below)Figure 21................................................................................................. 22 Schematic of camera and breakout board with Arduino (shown below)Figure 22. ......................................................... 22 Connections ...................................................................................................................................................................... 23 Connecting Camera and Arduino .................................................................................................................................. 23 Connecting SD breakout board and Arduino ................................................................................................................ 23 Power Supply Connection ............................................................................................................................................. 23 Getting Started...................................................................................................................................................................... 23 Software Needed: ............................................................................................................................................................. 23 Libraries Needed: .............................................................................................................................................................. 23 Testing Components Individually .......................................................................................................................................... 24 Testing SD card.................................................................................................................................................................. 24 Testing Camera ................................................................................................................................................................. 24 Store photo data into SD card........................................................................................................................................... 26 Putting It All Together ........................................................................................................................................................... 30 Step One............................................................................................................................................................................ 30 Step Two ........................................................................................................................................................................... 32 Future Considerations ........................................................................................................................................................... 36 Survey.................................................................................................................................................................................... 37 Processing ......................................................................................................................................................................... 37 Cameras ............................................................................................................................................................................ 37 NTSC/PAL .......................................................................................................................................................................... 38 Android ............................................................................................................................................................................. 38 Future Research .................................................................................................................................................................... 39

Page |3

Executive Summary
The Camera group is responsible for measuring distance to and tracking a target using a low cost CCD camera. The current onboard data subsystem is to be designed around an Atmega328P Arduino Uno microcontroller board. The group must use a camera during mission to track targets using edge detection.

Project and System Level Requirements
The top level requirement is to have the robot be completely autonomous using the camera to complete mission.

System Level Task Descriptions
Phase ONE Research articles on image processing (w/ edge detection) and color tracking Run through processing tutorials with jpg files Get camera to communicate to Arduino

Phase TWO Use video from camera to demonstrate edge detection and color tracking

Phase THREE Write color tracking and edge detection algorithms in C++ targeting the Arduino Uno microcontroller board

96 kB http://www. ha2pyFaduiZABvG4cWGliztKnOI%252bnXJLcGrUqiduMw%3d 3.bing.99 $3.50 $47.95 $44.3V to 5V translation http://www.complete datasheet available http://www.diydrones. 84 MHz clock speed.htm .com/Arduino-DuemilanoveBoard/dp/B004A7L3NC/ref=pd_sim_pc_2/175-6461241-7454258 $44. 50mA (@12V).complete datasheet available 5V to 15V input.com/products/10061 .50 1 CMOS Video Camera 640x480 $31.95 Two products = $80 || ~$20 for shipping .sparkfun. Product Description Source/Link Price Each Price Total Current Products for 400D 1 LinkSprite JPEG Color Camera TTL Interface capture and output JPEG images through UART https://www.99 TOTAL: ~ $50 Future Products for 400D 1 Arduino Due 512 kB flash memory.3V device . switchable NTSC and PAL output using a jumper http://store.95 FREE 1 SD Breakout Board does the 3.00 $47.com/Mini_high_resolution_CCD_camera_ide al_for_FPV_day_p/cm-26n.com/images/search?q=SD+Card+Breakout+Board&view= detail&id=CBEFE685D401083CBBCFBC4441BB71B32539C7BC $3.mouser.Full spec sheet available TOTAL : ~ $100.com/ProductDetail/Arduino/A000062/?qs=%2f SRAM.95 $31.Page |4 Cost Qty.95 1 Arduino Duemilanove Microcontroller board based Board on the ATmega168 $19.amazon.

For example. Green. In this example. white is 255/255/255 and black is 0/0/0.cmucam. Test your knowledge by introducing color tracking algorithm written in processing. Run through processing tutorials with jpg files. Figure below shows a sharpened area of an image done with convolution using a 3x3 matrix. These are merely simple ideas but show you how they can be built upon or improved with time and experience. searching for these specified pixels one by one and creates a bounding box and a centroid. 3. Read articles on image processing Color tracking is described in this article as isolating a single color to obtain useful information about that object. Processing is the language this article refers to. CMUcam4-color tracking. Using video from our hobby cameras demonstrate edge detection and color tracking. 2. Image pixels are shown to be in arrays and are easily accessed individually as well as by groups to produce different kinds of filters. sharpening and blurring images. display and filter images are discussed. RGB color channels have values with a range from 0 to 255.org/projects/cmucam4/wiki/Color-tracking_Explanation Figure 1. Having six thresholds allows a range of colors for slight variation in the color you are tracking due to lighting and shadows in an image. and the center pixel value is nine. an algorithm goes through the image once. such as edge detecting. Maintaining six color thresholds for each of the scanned pixels allows an upper and lower allowed value for each of the Red. Images and Pixels. Any single colored pixel will have a mixture of these three channels with each having a single value. Read articles on image processing (w/ edge detection) and color tracking previously mentioned. are discussed in a simple explanation and introduction to help with their understanding and coding. How to load. A noise filter is also used to improve accuracy. and Blue values that make up a single pixel’s color. where the eight weighted pixels that surround a center pixel are at a negative one value.Page |5 Technical Divisions Color Tracking and Edge Detection Track 1. http://www. the title of this article. .

http://www. In 2001. It is open source which makes it popular amongst professionals and students alike. along with the number of libraries available.edu/~hill/ee444/Lectures/Software.org/learning/topics/edgedetection. It is based on the Java Programming Language.pdf The link below takes you to an edge detection example shown in the figure below.Page |6 http://processing. and GNU/Linux platforms. Research Processing as a language and download environment to PC Processing is a programming language developed at MIT for nonprogrammers interested in visual arts. http://processing. This edge detection code will be discussed later on in the paper.html Figure 3. From there you have a choice of many examples. It can also be reached by clicking on the Learning tab from the Processing homepage. Mac. Libraries are accessed under the Reference tab while Tutorials and Basics (basic examples) are also under the Learning tab. Software is free to use and download. and clicking on Topics. Online community support continues to grow.org/learning/pixels/ Figure2. .csulb. It can be used on Windows. Ben Fry and Casey Reas founded Processing while students at MIT. Processing-edge detection example. Processing-tutorial on images and pixels.

http://processing. If you take the difference in brightness of two neighboring pixels you shall get a simple form of edge detection. http://www.org/ The first step is going to the Processing. shown below. From there you can access the Download page by clicking on Download or Download Processing. Click here . Differences in brightness will be due to an edge. Figure 5. Processing home page . tutorials. etc.com/examples/chapter-15/example-15-12/ Figure 4. Sunflower edges example for processing.Page |7 While learning and becoming more familiar with Processing and edge detection I studied this simple edges example from the link below. The figure shows how edges are found on an image of a sunflower.Access to download. How to Download and Install Processing In this section I will provide you with step-by-step instructions for installing Processing on your computer.learningprocessing. The link to the home page is provided below.org homepage. whereas a very small difference lends to no edge or discontinuity.

Page |8 The second step is once you are on the download page you can choose your operating system. Getting Started Tutorial . Getting Started tutorial helps to get processing running on your PC. topics. and getting started. Under Learning Tab.org/learning/ Figure 7. Download Step three is to figure out how to get started with simple examples to verify everything is working properly.Different tutorials and examples on basics. It also introduces you to the Processing Development Environment functions. http://processing. You can access Tutorials under the Learning tab or straight from the Download page.org/download/ Figure 6. There is a troubleshooting page if necessary. This tutorial will guide you through extracting files if necessary and running a simple example. Download Processing to your PC. Click on your choice of platforms and begin to download. Once there you can click on the Getting Started tutorial shown below. http://processing.

the toolbar. Figure 8. a one line message area. From top to bottom you see the name of your sketch.org/learning/topics/edgedetection. Proceed with using examples and/or modifying code Edge Detection example code The following code provides an edge detection example of a branch of leaves. Originally. The original code can be found here: http://processing. followed by a console for technical details. the center value in the kernel is a positive 9. The kernel values are adjustable and can be tinkered with to provide better results for each image. This is a general sharpening kernel. text editor. The third and fourth loop go through the kernel’s rows and columns and sharpen the image.Page |9 Example of Processing Development Environment Below is an example of the Processing Development Environment you would work with in Processing. The first for loop goes through the rows while the second for loop goes through the columns. By manipulating the code somewhat we are able to see a before and after shot of the leaves. Processing Development Environment (PDE) example .html . tabbed sketches. the menus.

for (int y = 1. This example is currently not accurate in JavaScript mode. // Multiply adjacent pixels based on the kernel values sum += kernel[ky+1][kx+1] * val. red/green/blue are identical float val = red(img. img = loadImage("leaves. } void draw() { image(img. sum). // Image is grayscale. 0). 7. // Displays the image from point (0.height. sum. { -1. 0). A convolution process takes place as each of the kernel values are used to calculate each pixel. // Create an opaque image of the same size as the original PImage edgeImg = createImage(img.width + (x + kx). 0. -1}. } } // For this pixel in the new image.jpg".pixels[] edgeImg.width-1. // Kernel sum for this pixel for (int ky = -1. } } // State that there are changes to edgeImg. . row by row.jpg"). // Loop through every pixel in the image. /* @pjs preload="moon. Through a process. x < img. each center pixel // The next line is needed if running in JavaScript Mode with Processing. */ float[][] kernel = {{ -1.width + x] = color(sum. -1}. set the gray value // based on the sum from the kernel edgeImg.0) img.updatePixels(). ky++) { for (int kx = -1. RGB). width/2. { -1.99. -1. PImage img.width.height-1. */ A Kernel is a 3x3 matrix that has different weights for each pixel. kx++) { // Calculate the adjacent pixel for this kernel point int pos = (y + ky)*img. -1.pixels[pos]). void setup() { size(640. // Draw the new image } Here is the algorithm used to sharpen each pixel. Four for loops go through the kernel column by column.P a g e | 10 /** * Edge Detection. img. ky <= 1. // Load the original image noLoop(). image(edgeImg. 360). kx <= 1. * * A high-pass filter sharpens an image. -1}}.loadPixels().pixels[y*img. x++) { // Skip left and right edges float sum = 0. y++) { // Skip top and bottom edges for (int x = 1. y < img. This program analyzes every * pixel in an image in relation to the neighboring pixels to sharpen * the image.js in this example is sharpened.

240).com/examples/chapter-16/example-16-11/ // Learning Processing // Daniel Shiffman // http://www. // Before we begin searching.learningprocessing. video = new Capture(this. Once the algorithm finds a colored pixel within a specified range of the color being tracked it will draw a specified sized shape around that pixel and thus track that color being targeted.start(). // XY coordinate of closest color int closestX = 0.10.loadPixels(). image(video.*.available()) { video. trackColor = color(10.read().0). smooth(). // Start off tracking for any color you input below video.video. float worldRecord = 500. Simple color tracking example code: The following code initially tracks a chosen color and can be changed by the user clicking on a particular colored pixel. // Begin loop to walk through every pixel . column by column. Original jpeg image is shown on left and filtered image on right shows the process of edge detection. // tutorial: http://www. } void draw() { // Capture and display the video if (video. This code captures video and in this example uses a laptop webcam. int closestY = 0.learningprocessing. } video.width.net/p5/libs/video/ // Variable for capture device Capture video.P a g e | 11 Edge Detection example shows edges of leaves of jpeg file: Figure 9.10). // A variable for the color we are searching for. color trackColor. The original code can be found here: http://www. frame by frame. The distance function is used to compare each pixel with the tracked color. the "world record" for closest color is set to a high number that is easy for the first pixel to beat. Two for loops go through each row.height).silentlycrashing.com // Example 16-11: Simple color tracking import processing.0. void setup() { size(320.

} } void mousePressed() { // Save color where the mouse is clicked in trackColor variable int loc = mouseX + mouseY*video. x ++ ) { for (int y = 0.pixels[loc]. strokeWeight(1. color as acceptable within range } of the tracked color. closestY = y. // This threshold of 10 is arbitrary and you can adjust this number depending on how accurate you require the tracking to be. chosen color is compared float r2 = red(trackColor). Click on the video feed to track that particular color. trackColor = video.P a g e | 12 for (int x = 0.g2.pixels[loc]. } Simple color tracking example shows tracking of blue color in center using webcam: Figure 10. stroke(0.b1.36).width. save current location and current difference A value of 10 is chosen to if (d < worldRecord) { worldRecord = d. float g2 = green(trackColor). Here is a snapshot taken as the sketch continuously tracks a shade of blue on a hat.closestY-18.g1. y < video. } } // We only consider the color found if its color distance is less than 10.100). y ++ ) { int loc = x + y*video. . accurately recognize the current closestX = x.width. float b2 = blue(trackColor).200).0.0). // What is current color In this simple color color currentColor = video. tracking algorithm the float g1 = green(currentColor).height.width. float r1 = red(currentColor). float b1 = blue(currentColor). to the current pixel color. // If current color is more similar to tracked color than // closest color. rect(closestX-18.36.255. x < video. // Using euclidean distance to compare colors float d = dist(r1.b2).r2. // We are using the dist( ) function to compare the current color with the color we are tracking. if (worldRecord < 10) { // Draw a rectangle at the tracked pixel fill(trackColor.

5 cam.pixels). above this threshold.v3ga.computeBlobs(img. // will detect bright areas whose luminosity > 0. void setup() { // Size of applet size(640.height. import blobDetection. 30*4). drawBlobsAndEdges(true. // Capture cam = new Capture(this. With this code a live video feed is used to detect blobs and computes a bounding box for these blobs. theBlobDetection. BlobDetection theBlobDetection. 0. } // ================================================== // draw() // ================================================== void draw() { if (newFrame) { newFrame=false. theBlobDetection.height).BlobDetection library import processing.width. A fourth input is deleted // ================================================== // setup() here from original code // ================================================== in order to run properly.quasimondo.setThreshold(0. 0. cam.net/processing/BlobDetection/.60).P a g e | 13 Blob Detection example code The following code is able to detect certain levels of brightness based off of a threshold chosen.height). image(cam.2f).setPosDiscrimination(true).read(). // . img.video. theBlobDetection = new BlobDetection(img. Once you download and import the library you have access to it via the File Menu under Examples. Off the Libraries page in Processing.width.true). } // ================================================== // captureEvent() // ================================================== void captureEvent(Capture cam) { cam. 480). Brightness threshold is input . it is necessary to download the library.*. Blob b. newFrame = true. First. under Computer Vision/Video. fastblur(img.0. img. 2).1 by Mario Klingemann <http://incubator. theBlobDetection.copy(cam.0.*. cam.height). img. img. boolean newFrame=false. From there you are able to browse the site or download from the download tab.width.Super Fast Blur v1.start(). } } // ================================================== // drawBlobsAndEdges() // ================================================== void drawBlobsAndEdges(boolean drawBlobs. PImage img. 0. Capture cam.width. there is a BlobDetection link that takes you to the following website: http://www. // BlobDetection here.2f. boolean drawEdges) { noFill(). 0. 40*4.com> // . be detected // img which will be sent to detection (a smaller copy ofBlobs the cam img = new PImage(80. // Comment the following line if you use Processing 1. will frame).

h)]. for(i=-radius.m++) { eA = b. gsum+=(p & 0x00ff00)>>8.255.i++){ dv[i]=(i/div). int r[]=new int[wh].xMin*width. .x*width. if (b!=null) { // Edges if (drawEdges) { strokeWeight(3). } } } } // ================================================== // Super Fast Blur v1. getBlobNb () gives the number of blobs detected. } } // Blobs if (drawBlobs) { strokeWeight(1). Algorithm adds a pixel and drops a pixel instead of recalculating the kernel for each pixel. Runs through to draw edges.0).i<=radius. int g[]=new int[wh]. int rsum. eB. eA. int[] pix=img.p.i<256*div. int b[]=new int[wh].y.getEdgeNb().int radius) { if (radius<1){ return.p2. rect( b.w*width.P a g e | 14 EdgeVertex eA. n<theBlobDetection. for (i=0.0).x*width.y*height.bsum.getEdgeVertexB(m). stroke(0. int vmin[] = new int[max(w. n++) { b=theBlobDetection.gsum.height.y++){ rsum=gsum=bsum=0. int hm=h-1. stroke(255.yMin*height.getBlob(n).i.width.y<h.p1. int wh=w*h.getEdgeVertexA(m).i++){ p=pix[yi+min(wm.0))].yp. Performs super fast blur!!! Blurs the image of each frame. bsum+= p & 0x0000ff.h*height ).m<b.getBlobNb() . if (eA !=null && eB !=null) line( eA. for (int m=0.0.x. int div=radius+radius+1.yi. int vmax[] = new int[max(w. } yw=yi=0. for (y=0.h)].1 // by Mario Klingemann // <http://incubator.quasimondo.yw.max(i. Runs through to draw blobs. } int w=img. eB.b. int dv[]=new int[256*div]. Draws the shape of a rectangle with certain parameters. eB = b.pixels.y*height ). rsum+=(p & 0xff0000)>>16. b. getEdgeNb () gives the number of edges detected. int wm=w-1.eB. int h=img.com> // ================================================== void fastblur(PImage img.b. for (int n=0 .

yi+=w.x++){ r[yi]=dv[rsum]. gsum+=((p1 & 0x00ff00)-(p2 & 0x00ff00))>>8.i++){ yi=max(0.y++){ pix[yi]=0xff000000 | (dv[rsum]<<16) | (dv[gsum]<<8) | dv[bsum]. yi++. if(y==0){ vmin[x]=min(x+radius+1. } yi=x. bsum+=b[p1]-b[p2].hm)*w. gsum+=g[yi]. g[yi]=dv[gsum].P a g e | 15 } for (x=0.0). bsum+=b[yi]. yp=-radius*w. bsum+= (p1 & 0x0000ff)-(p2 & 0x0000ff). rsum+=r[p1]-r[p2].x<w. for(i=-radius. } } } . } p1=x+vmin[y]. } for (x=0.y<h.x<w.i<=radius.wm). vmax[y]=max(y-radius. vmax[x]=max(x-radius.x++){ rsum=gsum=bsum=0. } p1=pix[yw+vmin[x]].yp)+x. for (y=0.0)*w. yp+=w. rsum+=((p1 & 0xff0000)-(p2 & 0xff0000))>>16. p2=pix[yw+vmax[x]]. p2=x+vmax[y]. gsum+=g[p1]-g[p2]. b[yi]=dv[bsum]. rsum+=r[yi]. } yw+=w. if(x==0){ vmin[y]=min(y+radius+1.

A gradual horizontal gradient from white to black would show smaller arrows pointing to the right. one for the horizontal and the other for the vertical direction. A bounding box in this case surrounds the darker areas. Code is run to detect blobs. Edge detection determines where intensity levels in brightness dramatically change. The following image is a snapshot of a live video feed using a webcam. Edge Detection Background Edge detection is a fundamental operation in image processing. Such detectors are considered Laplacian edge detectors. It reduces unnecessary data to be processed while maintaining structural integrity of an image. A matrix when following this form has a center which is used for a better convolution of the image. the 3x3 matrix kernel processes a center pixel with eight surrounding pixels. 5x5. . it can also be the absolute value of the sum of the absolute values of each. you can also find zero crossings to pinpoint where edges are located. An image changes its brightness over a series or range of pixels. Larger arrows would indicate a greater rate of change and the likelihood of an edge. in a one dimensional image the first derivative taken of a brightness intensity curve will give you a peak at the steepest slope (fastest change in intensity) which represents an edge. For example. so to eliminate as much noise as possible there shall be a threshold to what is acceptable as an edge. or gradient magnitude. boundary areas of brightness and darkness. Uses include tracking and recognition.P a g e | 16 Blob Detection example shows the detection of darker areas of image using webcam Figure 11. Using the second order derivative. used to give an approximation of a derivative when convolved with the image pixels. Two kernels can be used to approximate derivatives. is then calculated as the square root of the sum of the two squares (x and y-direction). therefore likely being an edge. or masks. Image gradient shows the direction of the intensity change from lighter to darker areas in a 2D image. Kernels. For example. The orientation angle gives you the approximate angle of the edge and is calculated by the arctangent of the vertical divided by the horizontal. Noise will definitely cause false edges. etc. The strength of the edge. Alternatively. are 3x3. The image gradient is only approximated when doing edge detection. The larger the matrix is made the better the approximation of the derivative. This is considered to be the image gradient and is what this edge detection is based on. matrices.

In general. Edge Detection Simulation in Matlab using webcam (Prewitt Method): In this demo. Prewitt Prewitt edge detector is based on first derivatives and has two kernels to approximate edges. In general. Canny in 1986. Laplacian Laplacian edge detector uses only one kernel and calculates the second derivative. one for horizontal and the other for vertical. below are not. Values above the highest threshold are edges. With this information it is possible to find the magnitude of the edge and the orientation angle of the edge. Sobel uses 3x3 kernels. the Video and Image Processing Blockset software is used in a Simulink simulation to detect edges in a live video feed using the Prewitt Edge Detection method. It also uses a threshold to produce the edges.P a g e | 17 Types of edge detection Canny The Canny edge detector was developed by John F. It figures out whether or not there are edges by the level of proximity to other edges. It runs the image through a Gaussian blur to filter out some noise and allow for a better image gradient.html . or search in the help section in Matlab: http://www. Based on the angle and magnitude. A Sobel. one for horizontal and the other for vertical to obtain the gradient of each point. Sobel Sobel edge detection is based off the first order derivatives but approximated with convolution of two kernels. and anything in between are dependent on the angle and magnitude of each pixel and others surrounding it. Set the Ports Mode to separate color signals in order to run. two thresholds are used to further filter for edges. Figure 12. or other algorithm is used to find edges. Shows the Sobel Kernels. Use the following link to open and run.mathworks. Shows the Prewitt Kernels. Prewitt uses 3x3 kernels. By clicking on the Edge Detection Block you have the option of using different edge detection methods along with the ability to set the threshold for the amount of edges detected. Figure 13.com/help/vision/examples/edge-detection-using-live-video-acquisition. it was necessary to adjust the From Video Device Block. Along with changing the threshold values.

. Edge detection shows the abrupt changes of brightness levels (edges) as a white line.P a g e | 18 Figure 14. Edge Detection example shows an image taken from webcam: Figure 15. Matlab Demo of Edge detection using Simulink. In a classroom environment this snapshot of a live video feed was taken.

To capture and process a picture. the internet and past projects/documents) discussed in Table 1. • • Default baud rate of serial port 38400 Hz Default baud rate for storing data in memory card is 9600 Hz • Table 1 .evola. a SD card breakout board (Figure 20) is required which can be plugged directly to the Arduino because it does the 5V to 3. www.3V translation.LinkSprite JPEG Color Camera TTL Interface • Camera allows you to capture and output JPEG images through UART • The camera requires a 3.P a g e | 19 Image Capture Track After an extensive review of the literature.com/datasheets/Sensors/Imaging/1274419957. We chose the LinkSprite JPEG Color Camera TTL Interface (Figure 16).csulb/edu/~hill/ee400d/Reference Folder/Camera/  Here are past projects from previous semesters.fr/product_info. an SRAM external memory module is required (SD card) since each JPEG image ranges from 4Kbytes to 47 Kbytes.sparkfun. (i.. Figure 16 . VGA/QVGA /320x240 resolution .pdf  Here is a manual on the camera provided by LinkSprite.edu/physcomp/sensors/Reports/Linkspritejpegcolorcamera  Download the JPEG Camera library www.Tutorials and Research Reports www. This camera will capture and output JPEG images through UART (TTL interface) and store the picture into a SD card. 1Kbytes of EEPROM and 2 Kbytes of SRAM memory. The Linksprite camera meets our objectives for a low cost CCD camera and also has a beneficial background with tutorials and research reports to reference.nyu.php/en/linksprite-jpeg-color-camera-ttl-interface-p-172  Specific tutorial http://itp. In order to use the SD card.3 or 5V power supply with a current consumption of 80 – 100mA. Given that we are using the ATmega328P which is a low-power CMOS 8-bit microcontroller with a memory size of 32Kbytes of Flash Memory.e.

slave out) – connected to pin 12 (this is the only input pin) SCK (slave clock input.3V. the following three output pins should not output more than 3.3V.P a g e | 20 As Figure 17 shows how small the camera is by comparing it to a US quarter. master clock output) – connect to pin 13 SS (slave select. To not damage the SD card logic. Using this board will do the power translation of 5V to 3. The camera is placed onto an OV706 chip that contains a Vimicro VC0706 chip (underneath the QC passed sticker) placed behind the camera as illustrated in Figure 19. Figure 17 Figure 18 Figure 19 SD card breakout board Secure-Digital (SD) card is used as a storage device. The Vcc of this board is 5V so we can plug it directly to the Arduino. This board contains only four pins that we will use in our module. The VC0706 is a digital processor specifically designed for image capturing and processing. this camera has a dimension size of 32mm x 32mm (Figure 18). The SD card and the SD module works on an SPI interface and only read/write single sectors (one sector means a 512 bytes block). active low) – connected to pin 10 MOSI (master out. QC passed means that a Quality Control Engineer approved the product before it was shipped out. MISO (master in. It contains JPEG embedded code which allows it to make real time coding for the capture image and its external control read the M-JPEG more convenient. slave in) – connected to pin 11 GND (ground) – connected to ground pin on Arduino VCC (Voltage connected between ground and the collector) – connected to 5V on Arduino . We must use this board because the Arduino has 512 bytes EEPROM which does not lose its contents at power off and 2048 bytes SRAM which contents are lost at power off.

Cards that are larger than 2GB must be FAT 32.org/downloads/formatter_3/  Right click on the card’s directory and choose “Format” from the drop down menu and be sure to choose FAT as the filesystem.P a g e | 21 Figure 20 – SD Card Breakout Board There are many different types of SD card to use. https://www. The SD standard specifies that 2GB and smaller should be formatted FAT16. Use FAT16 when possible because the process is pretty straightforward. but two that have no problem communicating are: SanDisk 2GB SD2 Card Apacer 60X 1GB SD Card. .sdcard. It is best to format SD cards with the SD Association’s formatter which will align file structures with the internal flash chips for optimum performance. File Allocation Table (FAT) File Allocation Tab le (FAT) is the name of a computer file system and a family of industry standard file systems utilizing it. Note: SD Card San Disk 16MB DOES NOT work.

P a g e | 22 Connecting Camera to Arduino (shown below)Figure 21. . Schematic of camera and breakout board with Arduino (shown below)Figure 22.

0.evola. active low) – connected to pin 10 on Arduino MOSI (master out.2 Libraries Needed: www.html  Download Arduino Code for breakout board www.cc/en/Main/Software  Download Arduino 1.php/en/linksprite-jpeg-color-camera-ttl-interface-p-172  Download Arduino Code and Library (Take a picture and save to SD card) .cutedigi. slave out) – connected to pin 12 (this is the only input pin) on Arduino SCK (slave clock input.connected to pin 4 on Arduino TXD (Transmitting Data) is connected to pin 5 on Arduino GND (Ground) is connected to GND on Arduino VCC (Voltage connected between ground and the collector) is connected to 5V on Arduino Connecting SD breakout board and Arduino MISO (master in.P a g e | 23 Connections Connecting Camera and Arduino TV out is not connected to anything RXD (Receiving Data) . master clock output) – connect to pin 13 on Arduino SS (slave select.fr/product_info. slave in) – connected to pin 11 on Arduino GND (ground) – connected to ground pin on Arduino VCC (Voltage connected between ground and the collector) – connected to 5V on Arduino Power Supply Connection Power connected directly to 5V and GND pin on Arduino and the other connect to breadboard Supply power for SD breakout board and camera Getting Started Software Needed: http://arduino.com/breakout-board/sd-card-breakout-board.

Select File. the output on the Serial Monitor will show the properties of the SD being used: Initializing SD card.count=0.2 has examples to run to see if the SD card is connected properly. SendReadDataCmd().P a g e | 24 Testing Components Individually Testing SD card Arduino 1. HEX values (FF D9) if((a[j-1]==0xFF)&&(a[j]==0xD9)) //Check if the picture is over EndFlag=1.read().begin(38400).available()>0) picture command { incomingbyte=mySerial. } byte a[32]. mySerial. void SendReadDataCmd().read().begin(38400).5).available()>0) { incomingbyte=mySerial. while(mySerial. wait 2-3 second to send take wait for four seconds to send take SendTakePhotoCmd(). Run the program and then upload. void SendTakePhotoCmd(). then Examples. //Configure pin 4 and 5 as soft serial port //Read Starting address void SendResetCmd(). while(mySerial. Lastly. while(!EndFlag) { j=0.. } in .Wiring is correct and a card is present. #include <SoftwareSerial. check the hex code printed starts at FF D8 and ends with FF D9.k=0. then SD which will show a list of example programs and select CardInfo. If so then connection with camera is valid (Figure 23). } void loop() { SendResetCmd(). count=0.By checking these values are printed helps confirms a JPEG file is being created. delay(25). j++.0. //After reset. k++.h> byte incomingbyte. Send reset command to camera and delay(4000). default baud rate of 38400 bps void setup() { Serial. int address=0x0000. SoftwareSerial mySerial(4. Initiates a serial communication at the void StopTakePhotoCmd(). boolean EndFlag=0. then Serial Monitor and verify on the bottom right of the box is reading the default baud rate (38400). if((k>5)&&(j<32)&&(!EndFlag)) Checks if the picture is done by ending { a[j]=incomingbyte.j=0. count++. Card type: SD2 Volume type is FAT16 Testing Camera Using this code will show that the camera is connected properly. Select Tools.. When the CardInfo example is ran. k=0.

} //Send jpeg picture over the serial port These are the HEX commands that are sent from the Arduino to the camera to: Reset Take picture Read data Stop camera //address increases 32 bytes according to buffer size .ML.HEX). mySerial. mySerial.print((char)0x00). mySerial. mySerial. } while(1). mySerial. mySerial.print((char)0x01). mySerial.print((char)0x36).print((char)0x00). mySerial. } void StopTakePhotoCmd() { mySerial.println(). mySerial.print((char)0x0a).print(a[j]. mySerial.print((char)0x26). } //Send Reset command void SendResetCmd() { mySerial. ML=address%0x100. } //Read data void SendReadDataCmd() { uint8_t MH.print((char)0x32).print((char)0x56). mySerial. mySerial.print((char)0x00).P a g e | 25 } for(j=0. mySerial.print((char)0x00). mySerial.print((char)0x03).print((char)0x00).print((char)0x00).print((char)0x00).print((char)0x00). mySerial. Serial. mySerial. mySerial.j++) { if(a[j]<0x10) Serial. } //Send take picture command void SendTakePhotoCmd() { mySerial. MH=address/0x100.print((char)0x0a).print(" ").print((char)0x36).print((char)0x0c). mySerial.print((char)0x00).j<count. mySerial. mySerial. } Serial. address+=0x20.print((char)ML). mySerial. mySerial.print((char)0x56).print((char)0x01).print((char)0x00). mySerial.print((char)0x00). mySerial. mySerial.print((char)0x56). mySerial.print((char)MH).print((char)0x00).print((char)0x00). mySerial.print((char)0x20).print("0").print((char)0x56). Serial.

SetImageSizeCmd().j=0.h> <sd-reader_config.h> <FAT16. mySerial. SendReadDataCmd().h> <fat. StopTakePhotoCmd(). Then take SD card out and put into computer to see if the picture uploaded as a JPEG file (Figure 25).h> <partition.ML.h> <sd_raw. SetBaudRateCmd().5). void void void void void void SendResetCmd().//Read Starting address uint8_t MH. SendTakePhotoCmd(). Repeat same steps mentioned in testing the Camera to verify hex code is printed (Figure 24).h> #include #include #include #include #include #include #include #include #include <byteordering. Run the following code provided and compile to make sure there are no errors. boolean EndFlag=0. //Configure pin 4 and 5 as soft serial port manipulate into the sketch long int address=0x0000.h> <partition_config.begin(38400).h> <sd_raw_config.count=0. void setup() { Serial. FAT TestFile.P a g e | 26 Figure 23 – Serial Monitor Checks if the picture is done by ending in HEX values (FF D9) Check default baud rate is being used Store photo data into SD card After verifying that both the SD card and Camera are working.h> <fat_config. i=0.k=0. It is now time to integrate both to work together.begin(38400). #include <SoftwareSerial.h> This will be the file we byte incomingbyte. if(!sd_raw_init()) { . SoftwareSerial mySerial(4.

while(1).write will always write to the } //Send jpeg picture over the serial port TestFile. } //Send Reset command void SendResetCmd() { mySerial. } // create a new jpeg file Initializes the void loop() { SendResetCmd(). Serial. k=0.print((char)0x56).P a g e | 27 while(1). //After reset.available()>0) { incomingbyte=mySerial. while(mySerial.open(). mySerial.print((char)0x00). beginning of the file } TestFile. mySerial. wait 2-3 second to send take picture command SendTakePhotoCmd(). delay(4000). SD card and the FAT file system then creates a JPEG file on the SD card while(mySerial.read(). count=0. delay(20). serial.print((char)0x01).print((char)0x04).j<count.j++) { if(a[j]<0x10) Serial.print((char)0x05).read().print will show us the file is done writing. mySerial. mySerial. i++.print("Finished writing data to file").jpg"). mySerial. Using . Serial.close().print(" "). } TestFile. mySerial. if((a[j-1]==0xFF)&&(a[j]==0xD9)) //Check if the picture is over EndFlag=1. TestFile.print((char)0x00). Since the file was created.print("0"). we can TestFile.print((char)0x00). count++. open and write to it while(!EndFlag) { j=0.HEX).print((char)0x56).write((char*)a). } void SetImageSizeCmd() { mySerial.initialize(). if((k>5)&&(j<32)&&(!EndFlag)) { a[j]=incomingbyte. k++. Serial.print(a[j]. Once HEX values are completed and printed out on Serial Monitor. } byte a[32]. mySerial.create_file("FiNAllY.print((char)0x31).available()>0) { incomingbyte=mySerial.print((char)0x26). j++. . mySerial. Serial. SendReadDataCmd().println(). } } for(j=0.

print((char)0x32).print((char)0x01). } //Read data void SendReadDataCmd() { //uint8_t MH.print((char)0x00). } //Send take picture command void SendTakePhotoCmd() { mySerial. mySerial. mySerial. mySerial. mySerial. mySerial. mySerial.print((char)0x00).print((char)MH).print((char)ML).print((char)0x00). mySerial. address+=0x20. mySerial. mySerial. mySerial. } void SetBaudRateCmd() { mySerial.print((char)0x36).print((char)0x00).print((char)0x56).print((char)0x00). mySerial.print((char)0x00).print((char)0x36).print((char)0x56).print((char)0x00). ML=address%0x100.print((char)0x01).print((char)0x00). mySerial.P a g e | 28 mySerial. mySerial.print((char)0x11).print((char)0x20). mySerial.print((char)0x03).ML. mySerial.print((char)0x0a).print((char)0x00). mySerial.print((char)0x00). mySerial.print((char)0x00). mySerial. mySerial. mySerial. mySerial. MH=address/0x100.print((char)0x00). mySerial. mySerial.print((char)0x00).print((char)0x0c). mySerial.print((char)0xC8).print((char)0x0a). mySerial.print((char)0x03).print((char)0x56).print((char)0x01). mySerial. } void StopTakePhotoCmd() { mySerial. mySerial.print((char)0x24). mySerial.print((char)0xAE). } These are the HEX commands that are sent from the Arduino to the camera to: Reset Take picture Set Image Size Set Baud Rate Read data Stop camera //address increases 32 bytes according to buffer size . mySerial.print((char)0x19). mySerial. mySerial. mySerial.print((char)0x56).

Final Output of the JPEG Image from SD Card .P a g e | 29 Figure 24 – Checking Serial Monitor for Completion Figure 25 .

eof = 0. } } } void takeAndSendPhoto(){ if(takingPhoto)return.TXT". } else if(inByte == 'T'){ takeAndSendPhoto(). address = 0. 25.begin(). int size=0. //Count is used to store the number of characters in the response string. It is time to put everything together using Arduino and Processing to upload the file directly to the computer.h> <SdFat.Send a byte to MemoryCard. int eof=0. //Create a character array to store the cameras response to commands char response[32]. boolean sendingPhoto = 0. if(inByte == 'S') { sendPhoto(filename).P a g e | 30 Putting It All Together Now that we have tested the SD. //establishContact().begin(). size = 0. unsigned int count=0. //while reading the file data from the camera. char filename[] = "/Images/IMG_01. count = 0. takingPhoto = true. // necessary? //Reset the camera . //Size will be set to the size of the jpeg image.available() > 0) { // get incoming byte: int inByte = Serial. 2010 */ #include #include #include #include <MemoryCard.h> - Must have all libraries included in Arduino //Create an instance of the camera JPEGCamera camera.h> <JPEGCamera.reset(response). void setup() { Serial. Step One Compile and upload this code into Arduino /* From Sparkfun.read(). the Camera and validated that a JPEG file is on the SD card. camera. //This will keep track of the data address being read from the camera int address=0. until receiver responds . boolean takingPhoto = 0.h> <NewSoftSerial. // send a byte to establish contact until establish receivercontact responds } void loop() { if (Serial. int takeNewPhoto = 0. delay(3000).com Modified by Adam Harvey Dec.begin(115200). count=camera.

delay(3000). MemoryCard. true).readData(response.open(filename). BYTE). } } MemoryCard. sendingPhoto = true. //Get the size of the picture count = camera. //Store all of the data that we read to the SD card for(int i=0. MemoryCard. count=camera. .print( (char) b . int c. // IMG char *imgDir = "/Images/".print('A'.print('EOF'). int b.read()) >= 0) { Serial. // send a capital A delay(300).file. //Save the data to the SD card MemoryCard.close().P a g e | 31 //Take a picture count=camera. address).exists(imgDir)) MemoryCard. } void sendPhoto(char filename[]){ if(sendingPhoto)return. } } - Address starts at 0. } void establishContact() { while (Serial. } Serial. BYTE).print('EOF'). while((b = MemoryCard.open(filename. character.print(response[i]. sendingPhoto = false. if (!MemoryCard. Serial.print( (char) response[i] . BYTE).close(). then get out } address+=count. get out of this loop and stop reading data if(eof==1)break. count=camera.If we found the eof //If we found the eof character. int filenum = 0. filename[13] = (char) (b+48). takingPhoto = false. while(address < size) { //Read the data starting at the current address. keeps reading data until we have read ‘size’ data of this loop and stop reading the data  EOF is a flag for the sketch to determine when the end of a file has been detected .file.makeDir("/Images/").takePicture(response).getSize(response.reset(response). BYTE). i<count. break.available() <= 0) { Serial. int b){ filename[12] = (char) (a+48). i++){ if((response[i] == (char)0xD9) && (response[i-1]==(char)0xFF))eof=1. if(eof==1){ Serial. MemoryCard. } void setImageName(int a. &size).

if( readData ) { while( myPort.115200 ). println( " bytes . { -1. print( readBytes )..readString() ). } } } else { while( myPort. -1}. a separate window will open (Figure 26) . PImage edgeImg.serial. byte[] photo = {}. myPort = new Serial( this. } } if(done) { // Check to see if image is loaded if (!loaded) { img = loadImage(". Boolean readData = false. 0). import processing. i++ ) { photo = append( photo.P a g e | 32 Step Two Upload this code into Processing. 9. hit any key on the keyboard to start capturing your image(Figure 26). Serial. edgeDetection = true. hit any key on the keyboard to output the final image (Figure 27) .available() > 0 ) { print( "COM Data: " ). println( Serial.list()[0]. } else { image(edgeImg.*. 0). boolean edgeDetection = false. } void draw() { byte[] buffer = new byte[64]. boolean done = false. float[][] kernel = {{ -1. when you see that the image has stopped. loaded = true. } else { // Check to see if we edge detected the image if(!edgeDetection) { edgeDetect(). Serial myPort.jpg"). 240). boolean loaded = false. // Variables for Edge Detection PImage img. the output image has completed Edge Detection (Figure 27)." ). buffer[i] ).list() ). Select Run. } Variables for Edge Detection Runs this code when picture has been taken . println( myPort. 0.readBytes( buffer ). String filename = "photo. void setup() { size(320. -1}. myPort.buffer(10). -1. When completed. print( "Read " ).available() > 0 ) { int readBytes = myPort. for( int i = 0.. i < readBytes. { -1. -1. 0. -1}}. } // image(img./photo.jpg".

pixels[y*img.width. photo ).height-1." ). println( "Waiting for data . ky <= 1. sum.width + x] = color(sum. set the gray value // based on the sum from the kernel edgeImg. ky++) { for (int kx = -1." ). y++) { // Skip top and bottom edges for (int x = 1.loadPixels(). // Kernel sum for this pixel for (int ky = -1. y < img. } } // State that there are changes to edgeImg. sum).width + (x + kx). RGB). println( " bytes . } Edge Detection Code provided from the Processing website .height.pixels[pos]). kx++) { // Calculate the adjacent pixel for this kernel point int pos = (y + ky)*img. myPort.. saveBytes( filename. // Image is grayscale.length ). // Loop through every pixel in the image. readData = true.. done = true.. kx <= 1. img. println( "DONE!" ).write('T'). x++) { // Skip left and right edges float sum = 0..P a g e | 33 } } // KEYBOARD CONTROLLER void keyPressed() { if( photo. // Multiply adjacent pixels based on the kernel values sum += kernel[ky+1][kx+1] * val.length > 0 ) { readData = false.width-1. for (int y = 1.updatePixels(). } else { myPort.pixels[] edgeImg. } } This code will run when key on keyboard has been entered // EDGE DETECTION FUNCTION (from Processing website) void edgeDetect() { img. print( "Writing to disk " ). red/green/blue are identical float val = red(img. // Create an opaque image of the same size as the original edgeImg = createImage(img. x < img.clear(). } } // For this pixel in the new image. print( photo.

P a g e | 34 Figure 26 – Processing – Separate Window to Capture Image A separate window appears Press any key to start capturing image Figure 27 – Final Output of Image After a key has been pressed. the Final Image will print out Verify that picture is DONE! .

P a g e | 35 Figure 28.The JPEG file and the Output on Processing Figure 29 – The Plaque .

The semester should start with clear direction just like what we received in week nine.diydrones.com/ve/  Project done with Arduino and video camera using the video experimenter shield/strong background .70 (availability is a real concern here). Shield: Your chip. The Due operates at 3. we recommend having a strong leader who understands how the camera works and has a thorough knowledge on image processing. May also want to add First Person View (FPV) On Screen Display (OSD) support. http://nootropicdesign. Camera: http://store.htm $31.95 Microelectronic: Arduino Due $45.3v so depending on the operating voltages of your chip we may or may not need some level shifters.P a g e | 36 Future Considerations For future camera groups.com/Mini_high_resolution_CCD_camera_ideal_for_FPV_day_p/cm-26n.

m1850&_trkparms=aid%3D222002%26algo%3DSIC.FIT%26ao%3D1%26asc%3D11%2 6meid%3D3017466659087745656%26pid%3D100011%26prg%3D1005%26rk%3D3%26sd%3D110718181469%26  Same camera that was provided at the beginning of semester.cmucam.com/itm/New-Mini-Wired-Pinhole-Color-CCTV-Security-Surveillance-SPY-Hidden-Nanny-Camera/150930775367?_trksid=p2047675.P a g e | 37 Survey Here we have provided and categorized helpful links and research we discovered throughout the semester.pdf  User manual for c328 camera http://www.com/profiles/blogs/ardueyemini-4-gram-sensor-prototype . Processing http://vimeo.pdf  Data sheet for c328 camera http://www.com/12251298  Arduino forum on blob detection Cameras https://www. but unsuccessful with it https://www.m1850&_trkparms=aid%3D222002%26algo%3DSIC.com/products/8667  We used this camera.ebay.com/products/10061  Are currently in process of getting this camera to work http://embeddedeye.sparkfun.com/datasheets/Sensors/Imaging/C328_UM.FIT%26ao%3D1%26asc%3D11%2 6meid%3D3017466659087745656%26pid%3D100011%26prg%3D1005%26rk%3D2%26sd%3D110718181469%26  Support/tutorials http://www.com/datasheets/Sensors/Imaging/C328.com/itm/4LED-Mini-Wired-Camera-Pinhole-Hidden-Spy-Nanny-Color-CMOS-Video-Audio-CCTV-Cam/261038889820?_trksid=p2047675.org/projects/cmucam4  Has everything.sparkfun. but cannot use this camera to get to our final goal http://www.sparkfun.ebay. but cannot use due to price and basically too easy to get to ultimate goal since everything is provided https://www.com/products/9334  c328 camera info.sparkfun. just with LED lights around the edges to see in the dark.sparkfun. retired http://www.

societyofrobots.diydrones.davidchatting.shtml  Cool project with a well written document NTSC/PAL The two predominant worldwide formats are NTSC (National Television Systems Committee) and PAL (Phase Alternating Line).htm  Has great tutorials provided.com/products/637?  Simple cmos camera old/retired http://www.com/arduinoeyeshield/  Project with documentations Android http://code.fatshark.com/p/android-object-tracking/  Website useful for tutorials on how to use an android and do object tracking .pdf  Camera with output (NTSC/PAL) http://www. https://www. http://store.google.htm  Can’t use.sparkfun. Must figure out how to convert analog video stream into a digital video stream http://www. Unsuccessful. then vice versa when converting PAL to NTSC.P a g e | 38  Found by the president. When you convert NTSC to PAL. PAL is higher in resolution (576 horizontal lines) and updates on-screen images at 25 times per second. We tried to network with this guy. While NTSC has 480 horizontal lines. but did not reply back to any emails or messages to find out where and how much the camera would be.com/Mini_high_resolution_CCD_camera_ideal_for_FPV_day_p/cm-26n. but updates images more frequently at 30 times per second.com/pages/cmos/cmos-spec.makershed. PAL video has a tendency to flicker. 480 lines of resolution has to convert up to 576 lines of resolution and 30 images per second down to 25 images per second. but did not use this camera http://www. while motion is performed better in NTSC video.com/electronics_blackfin_camera.com/TTL_Serial_JPEG_Camera_with_NTSC_Video_p/mkad23.

95 Microelectronic: Arduino Due $45.com/Mini_high_resolution_CCD_camera_ideal_for_FPV_day_p/cm-26n.htm $31. Total will be around $100.cc/en/Main/Buy was sold out or on back order until mid December. .diydrones. May also want to add First Person View (FPV) On Screen Display (OSD) support. Shield: Your chip. The Due operates at 3. Camera: http://store.00”  Could not order Arduino Due because it was a brand new product and every website listed on http://arduino.3v so depending on the operating voltages of your chip we may or may not need some level shifters.70 (availability is a real concern here).P a g e | 39 Future Research Here is the solution which I believe gives us the most interesting camera with onboard processing support for the future.