Recent software development progress

1. Data-analyzing program

Last time we have brought progress to our data-analyzing program - we included waveform diagrams showing the trend of the change in input data from the eight sensor ports for further analyzing work.

These days we've been working on the methodology of music function triggering. Corresponding to our testing result before, area is one of the major factors affecting the received data value. Even with the same gestures carried out, the received values could vary a lot for different people as they have different hand shapes and masses.

As a result, we switched from our initial idea of direct data value analysis to a whole new approach - to differentiate the input data waveform under each small time period, so that the relationship of slope against time is found. This is useful as in this case we are observing and analyzing only basing on the actions of the users - whether they are approaching or leaving the sensors, whether they are generating a fast or slow beat; all these cause changes in slopes and the analyzing of the slope trends is accurate even with different users and different hand shapes.

Our re-developed UI

2. DJ table UI with several completed music effect

Meanwhile, we're also developing the user interface for the DJ table. At this stage we already have a simple interface that allows users to load in (.wav) songs from their local directories. Users can adjust the volume when playing the song. We also provided users with six parameters to adjust (Dry, Wet, Feedback, Sweep Rate, Sweep Range and Frequency). We also included some rhythmic music pieces to be remixed with the played music for a greater entertainment effect.

Our DJ table UI and the programming code

We will continue to work on both areas to develop the sensing methodology and improve the functions of the DJ table. Fighting!

Reference: Programming Audio Effects in C#

http://www.codeproject.com/Articles/3382/Programming-Audio-Effects-in-C

Progress slot 2: further board testing, data-analyzing program development and music effect testing

1. Further board testing

We proceeded to work further on board testing after we've finalized our project idea, in order to build up individual sensing parts to be combined into the DJ table later on.

The OpenCapSense board with the eight ports connected to respective sensors

In our project, we are going to make use of the maximum of eight sensors altogether for the gesture sensing and parameter controlling. Unlike in the past, we this time fully utilized the ports to develop a best motion sensing effect under the hardware limitation.

Four of the sensors

Testing of the motion sensing effect

2. Data-analyzing program development

Whilst we were working on the hardware part, we were in a mean time also developing our program for the data mining and data analysis. We realized the limitations of our current program - it reads data instantaneously from the eight ports and updates the data values in real time. However, with only raw data, it is impossible for us to observe any meaningful trends/ draw any conclusions. Besides, it is also not accurate to link user actions to the increase/ decrease in amplitudes of the waveforms as the change in waveform amplitudes are affected by a groups of factors (e.g. surface area of the approaching object, climate). We understand that if we want to find out the relationship between waveform patterns and the actions of the users, the only way is to relate the actions to the change in pattern trends instead of the change in absolute values (i.e. to analyze in an analog manner).

This is why we worked to optimize our program - we wanted to include real-time data analysis to be shown in line graphs in order to analyze user actions and assign meaningful system reactions to create music effects.

UI of our latest program

With the new function on the bottom part of the box, we can now directly find out the change in trends when a certain action is carried out. This is going to be crucial to our project development.

3. Music effect testing

Apart from the hardware and analyzing parts, we've also been working on some music effect testing. We wrote C# programs to create a UI that allows importing and playing of songs. We also included the volume changing function in our program, and it succeeded when we tested it on our OpenCapSense kit.

OpenCapSense FYP Finalized Application Idea

OpenCapSense FYP Application

DJ Table

Objective

After second thought, we decided to amend our project idea to apply the techonology into doing something interesting. We thought of applications in different areas, for the youngs and the elderly, used indoor and outdoor, for individuals and groups of people… Our only concern is that the idea can make use of the strengths of the opencap sense kit to a greatest extent.

We finally came up with the idea of a DJ table, which is roughly the same by means of function when compared with those you can find from the market. But our idea differentiates itself from all the other similar products in terms of controlling method. We are combining the capacitance sensing property of the kit with the various music effect a DJ table can create - instead of simply adjusting the knobs, we wish to present a sening table that allows users to do air-float gestures to trigger changes in parameters such as the adjustment in volumes, fast-forwarding and echo effect.

What is special in our product is that it enables effects to come with the natural body movements of humans when they listen to music, like the unconscious making of beats and the natural dancing-like movements. We believe that our OpencapSense kit application can bring the user experience of the DJ table to a higher level, and create more fun for the parties!

Details of the project

Since the OpenCapSense kit has its own limitations, we can at most connect to eight sensors at one time. It thus may not be that ideal if we assign one sensor to be responsible for one single function. In order to allow more possibilities of functions and gestures, we tried to correlate the sensors such that the functions are triggered by a series of sensing activities instead of the change in one single sensor. We came up with a number of prototypes as shown below:

Prototype A

Prototype B

Prototype C

Prototype D

Prototype A shows the simpliest assignment of sensors - each one is responsible for one single action.

Prototype B, C and D allows switching of controlling parameters with gestures (more than one sensors carrying out multiple sensing at one time to detect a particular set of body movements).

After a series of testing, we finally decided to adopt prototype D for our sensor assignment as we believed that such an arrangement can provide the users with the most freedom in gesturing.

We haven’t finalized the functions of our DJ table yet, but at this stage it will include some basic features such as volume control, play-forward/play-back function and fade-in/fade-out effect.

OpenCapSense FYP Application Idea 1

OpenCapSense FYP Application

OfficePro

Objective

Living in a metropolis with high-end technology around us, computer seems to have become a necessity to every one of us. Though not foolproof, Hongkongers are sitting in front of the monitors a few hours every day. With such a high usage this means it would be highly beneficial to the citizens if we walk a further step to improve the user experience - which is why we are introducing the OfficePro to you.

Our sensing table follows three main principles: environmentally-friendliness, convenience and fun. We allow automatic turning on and off of computer and lighting systems by detecting presense of users. This prevents unnecessary usage of electricity which helps to reduce greenhouse gas emission.

With the help of the OpenCapSense Kit, we are transferring the traditional keyboard onto a smart table which users can have complete control of the computer with simply some gestures. The table serves as a floating keypad that allows arrow and volumn control to be used for from basic internet surfing to word file editing to even pc game playing. It replaces part of the function of the traditional keyboard. It is modern and user-friendly, and it is fun.

Details of the project

Our sensing table will make use of the OpenCapSense Kit to serve functions that provides the users with a whole new working experience. Whilst the table brings convenience to users, we try to keep everything simple so that it is easy to use even for first-time users.

The sensing table basically serves the following functions:

1.    Automatic system turn on

   
When user sits on the chair and approaches the table, the action triggers sensor A to turn on the whole system. System recognizes that the table is in use and thus turns on the computer monitor and the light for the users. The remaining sensors also switch to the stand-by mode.

2.    Floating arrow keys

We understand that sometimes it’s just hard to do internet surfing with greasy hands of snacks, which is why we are designing the floating arrow keys to allow computer control without the trouble of having to touch the pad. Sensors B,G,F and H are by default set as UP, DOWN, LEFT and RIGHT keys respectively. This gives users more freedom to explore on the unlimited internet world and brings more interest to playing computer games as the sensors serve also as a game controller.

There is also a sensor C in between the four sensors for user customization later on. Users can decide the function of the key according to their individual needs.

3.    Volumn control

On the left side there lies sensors D and E which are responsible for the volumn control of the computer. D and E represent maximum and minimum volumn respectively, and users can move their hands in between the two to have gradual increase/decrease of volumn as the two sensors can sense change of input data due to movement of hands.

4.    System locker

A sensing table is somehow just a table, and there are times when we do not want any          additional features and want to keep it simple and plain. In this case, a locker may help. We decide default locking pattern for the system triggered with sensors B,F,G and H. It is easy like unlocking your smartphone. The only difference is that you do not even have to touch.

After a successful recognition of the locking pattern, any further control options would be denied. At this time users can do things on the table freely without needing to worry that their gestures would trigger some unwanted options for the computer. They now can do writing, drink cola or simply take a good nap on the table!

Progress slot 1: IKVM and Weka

Recently we have been working on a number of things to prepare for the start of the project.

1. Enhancing the board data collection performance

Last time we developed a C# program to display the collected data. But soon we found the program reacting a bit too slow that sometimes it failed to handle the large amount of input data, resulting in the program not responding.

Thus, we amended the way the data displays to enhance the speed of the program. At first, we put each of the input data into a list and display them accordingly to the user. The weakness of this method is that the system fails to handle the tremendous amount of data. Considering that it is not necessary to study all the information, we changed the way the data is displayed. This time we choose to extract and display at regular intervals so that not all information is shown.

2. Testing of the sensing board with variations in parameters

We tested the sensing board with different board size and distance and the nature of materials nearby (conduction/insulator).



Materials tested: 1. large sensing board; 2. small sensing board; 3. aluminium foil; 4. lead foil.

 We found that there are no observable differences in data values for different board sizes. But with a larger sensing board it increases the effective area that can be sensed.

For nature of testing materials, we tried to use the above materials to serve as sensing boards. We used bared hands to approach and finally touch the materials. We found that conductors (the foils) led to variations in data values, while insulators (wooden ruler) did not make observable changes in values.



A small sensing board



A large sensing board



Lead foil as a sensing board

The data value started to vary as the hand was around 15cm to the board. As it continued to approach the board, value increased. Finally when the hand touched the board, there was a significant increase in value immediately.



We connected two sensors in coils. This allows two independent sensors to sense a same particular area.

3. IKVM + Weka

We wrote lines to call out Weka for use in our C# program. However, Weka is compatible only with Java, thus cannot directly communicate with C#. We thus use IKVM to allow use of Weka on C#.

Reference:

IKVM with WEKA tutorial
http://weka.wikispaces.com/IKVM+with+WEKA+tutorial

C# code to run weka classifier
http://stackoverflow.com/questions/9820348/c-sharp-code-to-run-weka-classifier

Hands on testing of the OpenCapSense Kit

So this is our first hands on experience with the OpenCapSense Kit, our exciting journey begins!

Before start, we first explored on the OpenCapSense website and read through the documentations to gain better understanding of the components and the principles behind.

We then downloaded the Virtual COM port installation and installed the VCP drivers.

We then installed PUTTY to configure the USB port to allow connection.

Once connected, the microprocessor works to receive data from the eight USB ports in a round-robin manner at regular intervals.

PUTTY serves as a platform to display the received data and show them on the screen accordingly.

We also wrote a C# program to display the data just as what PUTTY does. For this program, we could save all the received data in a .csv file for later use.