Detecting fish on Raspberry Pi

After simulation on computer, I need to detect the fish on Raspberry Pi. In the case, the instruction for compiling C++ code with OpenCV library and WiringPi library is shown as following,

$g++ main.cpp -o main `pkg-config --cflags --libs opencv` -lwiringPi

And run it using following,

$sudo ./main

The results are shown as following video, I use the fish picture which is taken from fish lab,

Video 1: Detect fish on Raspberry Pi

As the video shown, it can detect the fish in the figure and then fish feeder will be rotating. Yeah, the screen also shows the status of feeder and if any fish is detected. 

This is an important step for project, and I nearly finish it, I think that I can consider the web control parts. 

Simulation for detecting fish on computer

Before detecting fish using Raspberry Pi, I need to detect fish on computer. In this case, I choose Windows 8.1 platform and Visual Studio for running the C++ code for detecting fish. The original image is taken from the fish lab in university, I choose one for detection,

Figure 1: Original image for fish

After processing, the picture is shown as follows:

Figure 2: Processed image

As shown in Figure 2, the color of background has been changed to the black and the color of noise changed to gray, the edge of fish has been changed to white, so that I need to count the number of white pixels which is shown in the window behind of processed image. In this case, the number of white pixels is 110. 

Then process another image which does not have fish as shown in Figure 3,

Figure 3: No fish in image

The processed image is shown as following,

Figure 4: Processed image

There are only noise on the processed image and the number of white pixels is 4. After statistics, the threshold value can be set 10. If the the number of white pixels are larger than 10, the feeder will be rotating because fish is detected, otherwise, the feeder does not operate. 

Install OpenCV on Raspberry Pi

If I want to process image, I need to install OpenCV library on the Raspberry Pi. I found a website which introduces the step teach how to install the OpenCV, it is shown as follows,

http://mitchtech.net/raspberry-pi-opencv/

Download, Install and config, It costs long time about several hours. You can follow every steps in above website.

A small C++ program to test library and GPIO Pins

I wrote a C++ program to control pins turning on or off, the program is shown as following:

Figure 1: C++ code for driving the fish feeder

A infinite loop is used to feed fish all time, and first output a high pulse from pin 11 to drive the feeder rotating for about 9.325s, which is the time for rotating one cycle. Then set the pin is low so that the feeder is idle. After that, set the pin being high again and so on. 

Then put it in the Raspberry Pi and compile it using the following command,

$g++ Blink.cpp -o Blink -lwiringPi

And then run it using the following command,

$sudo ./Blink

The results can be shown in following video:

Video 1: Video for controlling feeder using C++

It shows that the feeder rotates successfully, so this can be used to control GPIO pins using C++.

Programming language for second part

I used Python in the first part which is control fish feeder using timer, however, I think I need to change C++ for feeding fish automatically, because there are much more sources for image processing in C++. 

Then I need the C++ library to control the Raspberry Pi, I choose "WiringPi" which is a special C++ library for controlling Pi, I want to use it to control GPIO pins. The website for "WiringPi" is shown as

https://projects.drogon.net/raspberry-pi/wiringpi/

The first thing I need to do is install this library on the Raspberry Pi, the steps is shown in the following website

https://projects.drogon.net/raspberry-pi/wiringpi/download-and-install/

After that, I think I can write a small program to test it. 

Arithmetic for the second section

I have finished the second section last year, but I have not update yet, sorry for that, I am busy in that period, interim report, C++ assignment, preparing for final exam and so on. So I am plan to update the achievement in a few days, I hope that you like it. Furthermore, I am developing the third and fourth section now, it is related to the Internet of things. The user can use the website to feed and monitor their fish. 

The first thing I need to introduce is the image processing. The objective of second part is to feed fish automatically when the webcam detects the fish. So the webcam needs to take photo for fish tank at fixed periods. Then the Raspberry Pi is used to process images which is taken just now and detects the fish. Therefore, I decide to use edge detection to find the fish. 

The basic arithmetic is shown as follows,

1. Smooth: Suppress the noise which exists during the period of taking photos as much as possible, however, the real edges cannot be destroyed after reducing noise. This step is significant to improve accuracy for detecting fish. 
2. Enhance the quality of image: Use the filter to sharpening the edge of the image, the possibility of detecting fish will increase after this step. 
3. Detect the fish: In this step, program should determine what kinds of pixels should be regarded as noise and which can be kept for detecting fish, therefore, a threshold value can be set for filter the noise and keeping the original edge of fish. 
4. Find the location of fish: The program will determine the detailed location for fish in terms of detected the edges. Furthermore, edge linking and thinning will be used in this step.

It may be boring, however, it is very important, because this theory supports feasibility of practical C++ code. Ok, some interesting things are coming.

Start the second level

I have started my second level of project from last week, and this is much more advanced that previous one. After being finished, the fish feeder will feed fish automatically if fish comes here, the feeder will rotate to feed fish, so I think it can be called artificial intelligence project.

In this level, the webcam board (2302279) needs to be added in the raspberry pi as shown in Figure 1. 

Figure 1: Webcam board for Raspberry Pi

Furthermore, the rough algorithm is shown as following:

1.      Take photos for fish tank to detect if is there exists fish.

2.      Process the image to analysis if there is any fish in photos.

3.      If there is no fish in photos, take photos continually.

4.      If there exists some fish, drive feeder to feed fish.

5.      After some time, take photo continually.

So, the most important step is that analysis the photos to detect fish. The image processing will be involving in the second level. OK, how to process image? I will use OpenCV (Open Source Computer Vision Library). OpenCV will provide basic structure for computer vision and image processing so that we can use some functions to analysis images, you can use Matlab, C, C++, Python and so on to program. Furthermore, it can be installed in Windows, Mac, Linux so it can be used in main system platforms. 

Figure 2: OpenCV Logo

Raspberry Pi controls fish feeders using timer

I have finished the first level successfully, I use timer which is in Raspberry Pi to control fish feeder automatically, for example, the fish feeder will be rotated once every 5 seconds.

The results are shown in the following video:

Video 1: Raspberry Pi controls feeder using timer

This is important for my project and I have finished the first milestone. The next level is that feed fish automatically, which means that feeder will be rotating automatically if fish comes here and in the view of webcam. So the next level is challenger and it involves image processing.

The way to solve the problem

Today, I have failure to control feeder using GPIO again, I think that the GPIO cannot drive the motor. Please look at this forum:

A topic related to drive a DC motor using GPIO

The output current is about 20 mA, therefore, it is too small to drive the DC motor and it's enough to derive the LED light. 

I need to buy a expansion board called GERTBOARD for Raspberry Pi, the picture of this board is shown as 

Figure 1: GERTBOARD

So, I want to use it to drive the feeder, I hope it can bring luck to me.

Amazing happens

Today, I tried to use program to control feeder which is connected with the Raspberry Pi, however, amazing happens.

First, I tried to connected motor to 3.3V output pin (pin 1) in Raspberry Pi, the motor rotates normally as shown in Figure 1.

Figure 1: Schematic diagram for connecting motor to pin 1

However, when I program it and change pin 1 to pin 11, the diagram is shown in follows:

Figure 2: Schematic diagram for connecting motor to pin 11

However, the motor stopped all the time, but when I change motor to LED, the light will be turned on. 

So I measured the voltage of pin 11, it is 3.3V, voltage of pin 1 is also 3.3V, so I do not know why. It is amazing I am so confused. I think that there is no problem in pin 11, motor or cables. Therefore, I need to test it and find a problem. I think maybe the terminal of cable which is connected to pin 11. maybe loose or connect to other pins. I will give an answer in the future. 

Use remote desktop to control Raspberry Pi

I will introduce how to use remote to control raspberry pi using remote desktop. Firstly, install xrdp in Raspberry Pi directly or using SSH. The command is following:

$sudo apt-get install xrdp

And then install the Remote Desktop Connection in your computer. Next, find your Raspberry Pi's IP address using command.

$ifconfig

Input this IP address in the blank as shown in Figure 1:

Figure 1: Input IP for  remote connection

After that click connect, the interface is like Figure 2:

Figure 2: Input username and password for you Pi

Ok, we can connect our Raspberry Pi successfully like Figure 3:

If you can look this familiar desktop, you can control your Raspberry Pi from remote desktop.

Use SSH to control Raspberry Pi

OK, let's think how to control your Raspberry Pi using your computer. Therefore, you do not need to connect keyboards and mice to your Raspberry Pi, you only need to control it using your laptop. I use my mac to control it, if you use Windows, the method is little different.

So the first thing is opening your Raspberry Pi and then open LXTerminal to input the following command:

$ifconfig

The following information will be shown as Figure 1, this information is related to the network of your Raspberry Pi. For SSH, the IP address is important. 

Figure 1: Display IP address in Raspberry Pi 

OK, now I know my IP address in Raspberry Pi, and then open your mac Terminal to input the following command.

$ ssh pi@Raspberry-Pi-Address

Notice: pi is the user default user name for your Raspberry Pi, if you have changed your user name, please replace it. And replace the Raspberry-Pi-Address to your IP address in Raspberry Pi as shown in Figure 2.

Figure 2: SSH interface 

OK, you can control your Raspberry Pi with using your laptop now, for example, control GPIO pins as shown in Figure 3:

Figure 3: Input command to control GPIO

OK, we can control it successfully, the result are shown in following figures:

Figure 4: LED is turned on

Figure 5: LED is turned off

OK, we have controlled Raspberry Pi successfully so far, and next I will shown how to control Raspberry Pi through remote desk.

Python programming for controlling GPIO pins

Raspberry Pi has 26 GPIO pins, and GPIO is general purpose input/output. So if I can control the GPIO in raspberry pi, I will use raspberry pi control the devices. In this project, I chose Python to control Raspberry Pi.

There is a library can be imported to control the GPIO pins called RPI.GPIO  or another one is wiringPi. In my project, I will use RPI.GPIO. 

For testing the output GPIO pins, first of all I will use LED to test it. If LED is turned on, it means high voltage, otherwise, low voltage. So I wrote a small program for testing the output of pins as follows (save as Led_py.py):

import RPi.GPIO as GPIO

import time

GPIO.setmode(GPIO.BOARD)

GPIO.setup(11, GPIO.OUT)

while True:

           GPIO.output(11,True)

           time.sleep(1)

           GPIO.output(11,False)

           time.sleep(1)

After that, when I run this program, the command is like this:

$ sudo python Led_py.py

Notice: We cannot run this program directly because we are not root, so we must use "sudo" like root.

The results are in the following video.

Video1: How to control Raspberry Pi

The LED will be turned on for 1 second, and then turned off for 1 second, so the program will run all the time expect I stop it.

Connect the Raspberry Pi to wireless network "eduroam" in University

My project is related to the wireless control with using Raspberry Pi, so connecting Wifi in university is an important step in my project. It is easy to connect my Pi to normal Wifi because I know the name of Wifi, so I only need to provide the password. However, the Wifi is a little different in the university because I not only need to provide the password but also need to provide the account. 

Therefore, I decide to use WiFi Config (Figure 1) in desktop to connect the Wifi.

Figure 1: WiFi Config Logo

Next, download the certificate from the university website and save it in the desktop. Then copy it to the file /etc/wpa_supplicant, but you cannot use right click to copy if you don't log in as root, so I suggest use "sudo" command.

$ sudo cp /home/pi/the_name_of_certificate /etc/wpa_supplicant

And then open the WiFi Config in the desktop and set it using the same way in website of university.

Figure 2: The way to set WiFi "endroam"

Therefore, I can connect the WiFi to the wireless network in the university, if you cannot, try to reboot it.

Cool Style, Cool Webcam

This is a new cool style to write bloggers, I use the iPad and Apple wireless keyboard to write blogger. It is so interesting as shown in the following picture.

              Figure1: New style to write bloggers

So this really fun, and the most funny thing is that my webcam is arrived last Friday. This is special webcam board for Raspberry Pi as shown in following:

             Figure2: Webcam board for Raspberry Pi

I will upload the test results later with using this webcam, so I hope you can wait.

About this Project

This is my Final Year Project in University of Liverpool, I always want to do the project related to the hardware and software at the same time. Yeah, this is my desired project. I hope that I can enjoy it in my final year.

This project will use Raspberry Pi with wireless to control device to feed the fish and camera to monitor them separately. Furthermore, Raspberry Pi may measure the temperature in water and control the heater to keep warm for fish.

I think when I finished this project, I can achieve the following basic function:

• Use Raspberry Pi to control fish feeder automatically, for example, feed fish every 12 hours.
• Use website to control Raspberry Pi (wireless) to feed the fish.
• Raspberry Pi can monitor the fish and upload the video to the server, and then use website to receive this signal to observe how fish is fine.
• Feed fish intelligently, the feeder will feed fish automatically if the webcam detects the fish.