HEAT SEEKING FIRE EXTINGUISHER

By

Frisco Sembel Tony Winoto

ECE 445, SENIOR DESIGN PROJECT FALL 2011

TA: Jane Tu

December 7, 2011

Project No. 14

Abstract The goal of this project is to build an extinguisher with the precise ability to track the fire or moving heat source for the demonstration purpose using a modified webcam. Basically, our device starts to run when the alarm system goes off. Afterward, it will scan whole areas for contours and compare it to the next frame whether the contour is moving or not. Once fire sources are detected, the information regarding areas and locations are stored and sorted based on size. Then, the fire sources will be extinguished from the largest to the smallest. The extinguisher will be turned on for 5 seconds for each angle of fire locations. After all the fire is extinguished, the device will return to its original position and wait for alarm system signal to start.

TABLE OF CONTENTS
1. INTRODUCTION ............................................................................................................................................. 1 1.1 Purposes ....................................................................................................................................................... 1 1.2 Features and Benefits .................................................................................................................................. 2 1.3 Specification ................................................................................................................................................. 2 2. DESIGN PROCEDURE .................................................................................................................................... 3 2.1 Pulse Width Modulation .............................................................................................................................. 3 2.2 Vision System ............................................................................................................................................... 3 3. DESIGN DETAILS .......................................................................................................................................... 5 3.1 Beagleboard (Microprocessor) and Operating System ................................................................................ 5 3.2 Panning and Tilting Mechanism .................................................................................................................. 5 3.3 Extinguisher ................................................................................................................................................. 6 3.4 Alarm ........................................................................................................................................................... 7 3.5 Voltage Converter Circuit ............................................................................................................................ 7 3.6 IR Camera .................................................................................................................................................... 8 3.7 Power Supply ............................................................................................................................................... 9 3.8 Vision System .............................................................................................................................................. 9 3.9 Main Control .............................................................................................................................................. 12 4. DESIGN VERIFICATION .............................................................................................................................. 13 4.1 Beagleboard (Microprocessor) ................................................................................................................... 13 4.2 Panning and Tilting Mechanism ................................................................................................................ 13 4.3 Extinguisher and Alarm ............................................................................................................................. 13 4.4 Vision System ............................................................................................................................................ 14 5. 6. 7. ETHICAL ISSUE ............................................................................................................................................ 15 COST ............................................................................................................................................................... 16 CONCLUSIONS.............................................................................................................................................. 17 7.1 Accomplishments ....................................................................................................................................... 17 7.2 Challenges .................................................................................................................................................. 17 7. 3 Recommendations ..................................................................................................................................... 17 9. REFERENCES .................................................................................................................................................... 18 APPENDIX A ............................................................................................................................................................. A.1 APPENDIX B ............................................................................................................................................................. A.2

ii

1. INTRODUCTION
The main goal of this project was to build an extinguisher that can detect fire or moving heat source in this case using webcam. Since we are not allowed to use any form of fire, we will use light bulb to test our project. We got the idea about this project from damages caused by sprinkler in event of false alarm or just a small source of fire. Common sprinkler will spray water all over the area and will cause great damage to electronics. Since our system will only apply extinguisher on the source of fire/heat, the property damages caused by extinguisher are expected to be greatly reduced. The project is mainly divided into two crucial subprojects which are vision system and pulse width modulation (PWM) for servo. After these two subprojects are completed, there will another subproject that will bring these two subprojects together as one. The PWM subproject involves connection between BeagleBoard and the servos which is linked through voltage converter(shifter). On the other hand, the vision system involves BeagleBoard and the IR camera. Both the vision system and pulse width modulation are software based, so in order to bring them together we will use them as functions that can be called in the BeagleBoard. The illustration how we connect our device is shown on Fig. 1 below.

Fig. 1. Block diagram

1.1 Purposes

The purpose of our project was to develop an extinguisher that is equipped with a modified webcam and servos. The modified extinguisher will be able to track and pinpoint the source of fire/heat and turn on the LED to indicate the triggered extinguisher. The main goal of this project is to minimize the damages caused by common sprinkler system. If our device does not detect the fire/heat in the image from the webcam, it will continue to scan until the whole area is covered.

1.2 Features and Benefits Features

Portable fire extinguisher system. Fast fire detection Low power consumption Modular system
Benefits

Extinguishes fire source right on the source Reduce damage cost in case of fire

1.3 Specifications

The device is able to locate heat sources and differentiate heat emitting objects and real fire sources. The device also extinguishes the fire sources based on size of the fire sources from the largest to the smallest. The fire detection algorithm is also able to detect a fire source in less than 50 seconds for each panning angle. It should also be able to withstand the operating temperature between 0-85oC1.

Based on the operating temperature of Beagleboard. Refers to Beagleboard-xM Reference Manual on the reference page.

2. DESIGN PROCEDURES
This project consists of two major subprojects which are pulse width modulation system and vision system. Both subprojects are software based projects which are coded in c++ format. We also used OpenCV library functions to access some features such as detecting contour edges and calculating contour areas. The pulse width modulation subproject involves understanding of servos such as duty cycle at which servos should stop moving and time required to do full rotation. Both subprojects will be connected using control algorithm which calls both subprojects as functions.

2.1 Pulse Width Modulation

For the pulse width modulation there are two ways to access and generate signal from the PWM pins from the BeagleBoard: first is by a direct access to the OMAPs processor general -purpose timer mechanism and second is by using a servo driver for angstrom distribution. The first choice required interaction with the timer register in Linux kernel module which we do not have the knowledge of. Thus, we used the other method which helps us a little by providing us the driver2 to interact with the timer register. This method still took us sometimes to get it up and running. To get the driver work, we need to compile other codes using OpenEmbedded and Bitbake3 to get executable file from a Linux host computer. Afterwards, the file is then copied to the boagleboard to get access to the PWM. The servos used for our device are HS7954SH which were chosen based on the amount of torque generated. Each servo can generate 20 kg.cm torques at 4.8 Volt which is strong enough to operate our device. The torque of the servos is the most critical part of the system, since there will be high backward momentum from the high pressure extinguisher. The servos also have the capability to turn 360o, this is necessary for angle turning.

2.2 Vision System

For the vision system, we modified a normal webcam's filter by adding additional negative film on the filter. We use a normal webcam since commercial thermal cameras are relatively really expensive and we don't think it is necessary to use such expensive and detail thermal cameras which are used in manufacturing or monitoring company which require high precision temperature measurement. We also chose Logitech webcam which is supported by Angstrom distribution on the BeagleBoard since some other webcams are not working on Angstrom.

The driver are available as open source package from Scott Ellis. For more information, refer Scott Ellis on the reference page. 3 Cross-compile environment. Facilitate developers to create a complete Linux distribution for embedded system.

Our other consideration beside cost is the fire detection algorithm memory efficiency. Initially, we used haarcascade technique for our fire detection algorithm. However, this technique has many disadvantages for our device. It requires a lot of sample images to get accurate detection of an object. The minimum sample images to get at least 70% detection is around 6000 samples. The samples will be compiled into xml file that will be used in main control code. Since the file consists of many samples, it requires a significant amount of memory to run. Thus, it is really slow to run this technique on BeagleBoard which only has 512 Mb RAM. Finally, we decided to use moving contour detection technique for our detection algorithm. It basically works by comparing contour location of current and previous frames. The process is continuous until the device returns to its original position. This technique is really suitable for our device. It runs faster than the haarcascade technique on BeagleBoard. Furthermore, this technique is also easier to implement since it does not require additional file that consists of sample images.

3. DESIGN DETAILS
3.1 Beagleboard (Microprocessor) and Operating System

The main control and process of our project rely heavily on the microprocessor on the system. Since the project involves real time image streaming and processing, we decided to use Beagleboard-xM as our microprocessor, considering time it takes to do the image processing algorithm. Beagleboard-xM is a low cost, low powered single board computer produce by Texas Instrument. It runs on OMAP3530 system with ARM-Cortex-A8 CPU. It delivers 1 GHz clock speed with 512 RAM. The board also comes with various peripheral connections such as USB port, Serial port, DVI-D, expansion ports and many more. Those various types on peripheral over a large range of interfaces that can be connected to our modules, infrared camera, servos and I/O signal from and to extinguisher and alarm system. Beagleboard also has a large and active community that will come in handy when stuck with problems. Because Beagleboard is a single board computer, it needs an operating system that will run the board. We choose Linux Angstrom Distribution because there are more information available on how to build custom angstrom distribution and access the ports on the board. This allows us to choose only the necessary package to be included on the operating system and that increase the performance of the board. A particular important package needed is OpenCv. OpenCv is a library of programming function for real time computer vision. This helps us simplify our image processing algorithm by allowing us to call some necessary functions to process the real time images from the IR camera. Such functions are capturing images, extract images per frame, save images and background extraction, etc.

3.2 Panning and Tilting Mechanism

The mechanism used for controlling the panning and tilting of the simple robotic arm is done using the expansion header on the Beagleboard. The OMAP35304 system offers four PWM pins that can be accessed trough available headers. For Beagleboard-xM, only three of the pins are available for PWM connection on expansion port, they are port 4 - GPT9_PWMEVT, port 10 - GPT11_PWMEVT and port 6 GPT10_PWMEVT. There are two ways to access and generate signal from the PWM pins: first is by a direct access to the OMAPs processor general-purpose timer mechanism and second is by using a servo driver for angstrom distribution. The first choice required interaction with the timer register in Linux kernel module which we do not have the knowledge. The other alternative helps us a little by providing us the driver 5 to interact with the timer register. This method still took us sometime to get it up and running. To get the driver work, we need to compile the codes using OpenEmbedded and Bitbake6 to get the .ko executable file from a
4 5

Texas instrument general purpose f ARM architecture processor. The driver are available as open source package from Scott Ellis. For more information, refer Scott Ellis on the reference page. 6 Cross-compile environment. Facilitate developers to create a complete Linux distribution for embedded system.

Linux host computer. The file is then copied to the boagleboard and run to get access to the PWM. There is another thing to do before the PWM works. By default the kernel7 version that is used on our system enable the kernel config option CONFIG_OMAP_RESET_CLOCKS. This is a default kernel config that help save power consumption on Linux. In order to the get the PWM running, the option needs to be disabled. It is done by modifying the deconfig file inside the OpenEmbeded recipes folder and recompiling the kernel. The servos used for the pan and tilt mechanism are HS7954SH. HS7954SH were chosen based on the amount of torque generated. Each servo generates 20 kg.cm torques at 4.8 Volt. The torque of the servos is a critical part of the system, since the there will be high feedback force generated by the high pressure extinguisher. The servos also have the capability to turn 360o, this is necessary of panning. Since these are continuous servos, to control the angle produced, we need to keep the duty cycle of the PWM signal constant, for constant angular speed, and time how long does it take for the servo to do a full 360o rotation at current duty cycle. The angular speed can be adjusted accordingly. The time is then used to calculate how long it takes for the servo to do one step rotation (capture angle of the IR camera). Our measurement showed that our camera is able to view ~37 degrees for panning and ~30 degree for tilting. For the complete view of the entire area, the camera needs to take 360 degrees view which means the camera angle will be turned 10 times until all area is covered.

3.3 Extinguisher

The extinguisher is simply a fire extinguisher tube with some modification on the release mechanism. The release mechanism works just like an automatic restroom faucet. When release signal is high, it triggers the valve on the tip of the fire extinguisher tubing. One thing to note is that for the purpose of demo, we are not going to use a real fire extinguisher; instead we are going to use LED to indicate the trigger signal is high or low. Compared to PWM, accessing the general-purpose input-output (GPIO) pin on the board is much less complicated. The GPIO pins are also available through some of the ports on the expansion header. There is no need for driver. The GPIO can be access using sysfs 8 from C code. The signal for the extinguisher is set to port 17 GPIO 132 of the expansion port.
7 8

Kernel is a bridge between applications and the actual date processing done at the hardware level. Virtual file system. Exports information about devices and drivers from the kernel device model to user space (Source : Wikipedia)

3.4 Alarm

Alarm system is what starts all the process of fire detection. When the alarm goes off, it sends signal to the control system to start look for fire sources. The control then sends a feedback signal to turn off the alarm signal. All these are done using the GPIO capability of the board. Unlike the extinguisher who only needs an output GPIO direction, the alarm system needs input and output GPIO direction. For the purpose of demo, the input signal from the alarm signal is replaced with a push switch and the output of the alarm signal is replaced with a LED. When being pushed, the switch shorts port 9 GPIO136 on the expansion port to the ground. The output is showed through the LED that is connected to port 19 GPIO 131 on the expansion board.

3.5 Voltage Converter Circuit

Unfortunately, beagleboard only output maximum 1.8 V on its expansion port, this includes PWM and GPIO signals. So, in order to accommodate the required pulse signal for the servos, which is greater than 4.8V, we need to have a voltage converter circuit that will elevate output voltage from 1.8V to 4.8V without altering the original PWM signal from the beagleboard. We built the circuit using a general purpose low-power NPN bipolar junction transistor, 2N2222 and some 1000 Ohm resistors. Fig. 2 and Fig. 3 show the schematic and the simulation of the voltage converter circuit. The simulation shows three values, the 1.8V output from the beagleboard, the amplified but inverted signal from the left transistor and the amplified signals from the transistor on the right. It can be seen that by using this voltage converter circuit will satisfies the input pulse requirement for the servos.

Fig. 2. Voltage Converter Circuit (1.8V-5V)

Fig. 3. Voltage Converter Simulation

3.6 IR Camera

For the IR camera, we modified a regular Logitech webcam. Originally the Logitech webcam capture RGB( Red, Green, Blue) spectrum and combine them to get color images. However, we want to maximize the fire detection using the infrared camera. Thus, the filter inside the Logitech webcam is removed and replaced with a black photographic negative which technically will block all other lights and only allow infrared radiation to enter the camera sensor. This feature helps us to eliminate other objects that do not radiate infrared from the images taken and maximize the hit rate or accuracy of fire/heat source detection. The camera will be powered by BeagleBoard and connected to the USB port on the BeagleBoard. Fig. 4 and 5 shows how we modified the filter inside the webcam by inserting the photographic negative strip in between the camera sensor and its optical lens.

Figure 4. Logitech Webcam

Figure 5. Logitech webcam with modified filter (circled in red)

3.7 Power Supply

For the project, we are using 5V@3.2A power supply adaptor that connects to the 120V AC voltage. This adaptor will be connected to the power pins of both beagle-board and the servos. Other than those, the other components are going to be powered by the beagle-board.

3.8 Vision System

The image processing algorithm is coded in c++. We use some OpenCV functions and libraries in the code to process the images from the camera. The algorithm mainly works based on the threshold values of images to identify whether the heat/fire source is spotted in the image. We initially used haarcascade technique to provide the algorithm with threshold comparison. However, this technique did not work well for detecting fire since fire does not have certain shape. Finally, we decided to use moving contour detection technique which is much more suitable for our case. The haarcascade xml file requires three types of samples: positive, negative, and natural samples. The positive images are the images that only contain the object we want; in our case, the object is heat source. Since we are detecting fire/heat source in our project, we are just going to use the threshold values from the heat source (light bulb) for the positive image (Fig. 6). The negative images are images that contain background images without the object as illustrated in Fig. 7. The natural images are images that contain both background images and the object shown in Fig. 8. The positive sample will provide the threshold for the heat source while the negative will give the threshold for the background which will be dark for most of the images since we are using IR camera.

Fig. 6. Positive Image

Fig. 7. Negative Image

Fig. 8. Natural Image

The hit rate is quite high (~70%) for the current algorithm and threshold value around 0.5. However, the test was conduct using light bulb instead of real fire. Since fire has unpredictable pattern, this method of detection will not be suitable in real life cases. Thus, we change our method of detection into moving contour detection. The moving contour detection also uses opencv library which provide functions that return value of edges of the contour. First of all, the first frame of the video will be converted into black and white
9

image by applying threshold to original image in Fig. 9. After the image is already in black and white, it is clearly shown when there is heat source in the image shown in Fig. 10. Then, we connect the edges of the contour detected illustrated in Fig. 11 and compare the location of contour with the next frame to detect movement. Moreover, the areas of each fire sources are also stored. These values will be used to sort the fire source from the largest to the smallest. Thus, our device will extinguish the fire source according this sorting. This method is used to distinguish the real fire from static heat source.

Fig. 9. Original Image

Fig. 10. Black and White Image

Fig. 11. Detected Contour Image

10

3.8.1 Vision System Flowchart

Fig. 12. Vision System Flowchart

11

3.9 Main Control

The main control is the main code that will call functions from the fire detection algorithm and the servos control to get inputs and give outputs for panning, tilting, extinguisher and alarm system. The control starts out by loading the PWM driver module to the Angstrom operating system. It then waits for the signal from the smoke detector alarm to be able to proceed to the next state. If the alarm signal is triggered by the smoked detector, the control will activate servos and the IR camera and start positioning the whole system to 10 pan angles and 3 tilt angles until it covers all the area below the ceiling of a room. While at each particular angle position, it will process each frame of the real time video stream from the IR camera using the image fire detection algorithm described on the vision system. If fire is detected, then the angle position and the size of the fire source are saved in arrays which will then be sorted based on the area of the fire sources detected. It will then extinguishes the fire sources based on the angle position with the largest area of fire sources first then to rest of the fire sources detected. After all fire sources have been extinguished, the system will go back to halt and wait the alarm signal.

Fig. 13. Main Control Flowchart

12

4. DESIGN VERIFICATION
4.1 Beagleboard (Microprocessor) Verification for the Beagleboard was more of qualitative processes. It started out by building the image of the Angstrom-Distribution OS with the necessary libraries or packages and porting it to micro SDHC card which will then be inserted into the Beagleboard to boot. The booting process could be seen directly on the monitor that was connected to the boardboard trough DVI connection. Errors during the booting process of the Operating System will be listed on the monitor. It took us some trials to get just the right packages for the OS to run and the OpenCV to compile9.
4.2 Panning and Tilting Mechanism

Testing the panning and tilting mechanism of the simple robotic arm was done by varying the duty cycle of the PWM signals through the servo driver to get just the right angular speed needed. The configuration for 50Hz pulse was 20seconds/360o for panning (running at 7.53% CW and 7.4% CCW duty cycle) and 3seconds/90o for tilting (running at 7.09% CW and 7.31% CCW duty cycle). The neutral duty cycle was at 7.21% for tilting and 7.45% for panning. To make sure the output duty cycles match the input, we connected the pulse generated by the beagleboard to an oscilloscope. The outputs are attached on the appendix A. Servo Duty Cycle (PWM pulse signals run at 50Hz)

Neutral % Clockwise % Counter CW % Panning 7.45 7.53 7.40 (20s) Tilting 7.21 7.09 (3s) 7.31 (3s) Table 1. PWM Signal Configurations
4.3 Extinguisher and Alarm

Verifying signals that goes and comes from the alarm system and the extinguisher was done by watching the signals generated by the Beagleboard GPIO pins using oscilloscope and see if they match the desired signals. Fig. 14 below shows the signal generated from the GPIO pin that is used to trigger the fire extinguisher which in this case is an LED. The wave form runs at frequency of 2 kHz with 50% duty cycle and maximum voltage of 2V.

We used references from angstrom-distribution.org and http://elinux.org/Category:ECE497.

13

Fig. 14 Extingusher's output signal when triggered.

4.4 Vision System

We used bright LED to test our fire detection algorithm by moving the LED around to imitate real fire or just put the LED static to illustrate static heat source. The moving contour detection technique has high detection rate up to 90%. Our device can clearly distinguish the moving LEDs and static LEDs. However, since our device is rotated around the room and the algorithm is continuously running until all angles are covered, there might be false detections when our device is rotated to static LEDs which reduce the detection rate. The average processing time for each angle can be maintained at 1 second without including time to rotate the device. Since it takes 20 seconds to cover 360o, the total time spent at certain angle is around 3 seconds. Thus, the whole time to cover entire areas is around 3 minutes max. This time calculation is measured without showing the processed images on the screen since putting the image will slow down the entire system. The average frame rate per second for the camera is also measured around 17 fps and it is below Nyquist rate of real fire flicker frequency by 3 fps. However, the overall detection works pretty high.

14

5. ETHICAL ISSUE
The original goal is mainly to minimize the property damages caused by sprinkler system. Despite a lot of effort put into developing the device so that damage is minimized, there is still a possibility that the device is considered causing damage by some people. In this case, our device can sprays water to electronics which is currently very near to the heat source or actually is burning. This case related to IEEE Code of Ethics 9: to avoid injuring others, their property, reputation, or employment by false or malicious action. At first, we have the robotic arm that moves our system throughout the entire room. However, because of the complex mechanical system of the robotic arms, we decided just to mount the device on the ceiling. Therefore, this will avoid injuring people since the device is static on the ceiling.

15

6. COST
Name Frisco Sembel Tony Winoto Salary($/hr) $40.00 $40.00 Hours 180(18hr/week) 180(18hr/week) Total Total $7,200 $7,200 +Extras(2.5) $18,000 $18,000 $36,000.00

Part Aluminum Plate Webcam Camera Infrared Filter HS-7954SH (Robotic Servos) Transistor 2N2222 LED Wires Switch Circuit board Resistor 1k Mini extinguisher Beagle-Board Power Adaptor 5V@3.2A

Cost $20.00 $25.00 $1.00 $100.00 $0.25 $1.00 $10 $1.00 $3.00 $0.20 $30.00 $150.00 $25.00 Total

Quantity 2 1 1 2 4 2 1 1 1 6 1 1 1

Total $40.00 $25.00 $1.00 $200.00 $1.00 $2.00 $10.00 $1.00 $3.00 $1.20 $30.00 $150.00 $25.00 $489.20

Labor Cost $36,000.00

Parts Cost $489.20

Grand Total $36,489.20

16

7. CONCLUSIONS
7.1 Accomplishments The standalone vision system which consists of fire detection algorithm has successfully detected light emitting objects and distinguished whether they are fire sources or just light sources in less than 5 seconds which is one of the specifications. The panning and tilting mechanisms respond accurately to the input commands with various duty cycles. These mechanisms are able to move the extinguisher to the right position angles that are specified. The system is able to take input from the alarm system (switch) and output to the LED when the extinguisher is triggered. 7.2 Challenges There were some challenges we encountered in integrating the whole project. The main control of the system processes and stores a lot of information when the system runs. Thus, a considerable problem on the overall processing time arises. On the specification, the system should be able to perform the fire detection process in less than 50 seconds for each tilt angle or one full panning rotation. In the end, the result shows slower detection time at around 1 minute for each tilt angle. Detecting fire sources in a very bright room also raises a problem. Since the camera uses physical filter (negative photographic film), the camera will see a wider range of objects when the room is too bright. Therefore, even a plain wall will appear as a light emitting object. This will confuse the fire detection algorithm and result in a bad decision taken by the algorithm. In order to create a modular system, we did not connect the subsystems permanently. Instead, we made connectors for each subsystem. During the demonstration, lose connectors between the beagleboard PWM ports and the servos caused the servos to behave unpredictably. We figured that the lose connectors caused unexpected spikes on the pulses for controlling the servos. 7. 3 Recommendations For future works of the project, we need to optimize the control system and the fire detection algorithm in order to speed up the whole process. Another way to speed up the fire detection process is to take advantage of the DSP chip on the beagleboard. Additionally, for precise angular control, we will replace the continuous servos with regular servos.

17

9. REFERENCES
[1] [2] S. Ellis. (2010, June 8). Omap3-pwm [Online]. Available: https://github.com/scottellis/omap3-pwm Beagleboard.org. (2010, April 4). BeagleBoard-xM Rev-B System Reference Manual [Online]. Available: http://beagleboard.org/static/BBxMSRM_latest.pdf [3] How to Install OpenEmbedded and Bitbake [Online]. Available: http://pixhawk.ethz.ch/wiki /tutorials/omap/openembedded_bitbake_installation Linux kernel, PWM and GPIO [Online]. Available: http://elinux.org/Category:ECE497 Robotzone, LLC. (2011). HS 7954sh Servo [Online]. Available: http://www.servocity.com /html/hs7954sh_servo.html K. Koen. (2011, March 21). The Angstrom Distribution [Online]. Available: http://www. angstrom distribution.org/ V. Pisarevsky. (2011, August 24). OpenCv [Online]. Available: http://opencv.willowgarage.com/wiki/

[4] [5]

[6]

[7]

18

APPENDIX A

A.1

APPENDIX B

A.2