|
Original Project Proposal
Week of:
4/16
4/9
4/2
3/26
3/19
3/12 3/5 2/26
2/19 2/12 2/5
1/29 1/22 1/15
4/16
Tasks to be Completed:
I had to get the number of regions to be specified by the user instead of hard coded. I was also going to work on
the means lighting up for normalized sequences like the raw sequences.
Progress on Tasks:
I got the regions to be entered in in the GUI display. Any number betweek 1 and 16 can now be entered for the
numbers for the means. This was a lot more complicated than I thought it was going to be because I had to edit
some code involving kmeans that was hard coded before at 8 regions. The normalized sequence stuff was giving me
lots of problems and wasn't nearly as straight forward as the raw sequence stuff was since I had been programming
all my stuff previously for the raw sequence data. I didn't get that done, if I have time later I'll try to
implement it. I also went and started commenting some parts of the code
I didn't before and tried to clean it up a bit. Finally, I set up a scoring scheme for
the right and left arm movements. I basically set up a simple formula which was
(# of regions hit/# of regions*100)-mean distance of test point to nearest
point in the cloud.
4/9
Went to the Center for Future Health opening. Couldn't work on anything though cause I didn't
get the keys or local access to the machine until the following week.
4/2
Tasks to be completed:
I had to start on a scoring system that set up a threshold that each test point had to be within in reference to the
nearest point in the point cloud. If it was within that threshold, then it could be considered as a valid point, and
then compared against that of the point cloud. I was also, if time allowed, suppose to start on figuring out a way that
the user could enter in the number of means that would be generated to be compared against the test sequence.
Progress on Tasks:
I ran into lots of complications this week that I didn't expect. I still got a good amount to show, but not as much as
I would have liked for the time I spent. I got the raw sequence means to light up in blue as you match them in the movement.
The norm sequence was a little more complicated, however. Since all of my computations up to this point have been with the raw
data it's going to be very difficult to nail down where each given point in the normalized sequence is in relation to the raw means
that were set up. I have an idea on how to do it, though I wasn't able to have it implemented for this week. The text box for
prompting for the number of means was also kind of a pain...especially since I really haven't had that much experience with the
GUI Windows Programming. I actually got mislead a little bit cause I found a bug in the code for the other text box that allows you
to enter in some invalid values and get away with it, so I had to figure out a way around that. As it is right now, I have the
text box give the number of regions into the program, now I just have to change all the code that had the number of regions hard
coded in. That will be kind of a pain, but it shouldn't be much of a problem to do.
3/26
Tasks to be completed:
Familiarize myself with OpenGL, use it to map the means in the backdrop of the movements.
Progress on Tasks:
A good amount of my time this past week has been spent going through the code to try to figure out how the
current OpenGL stuff is implemented. This proved to be more complicated than I expected because I ran
into some Timer functions that called certain functions independently depending on if it was a normalized
or raw sequence, and there were lots of helper functions I had to track down before I realized most of what
was going on in the code. Once I got a firm handle on what was going on, I made the "Play Mov." button do
basically the same thing as the arrow button does now, basically to play back a sequence. It wasn't just
calling the same function twice, however. I made all new variables and flags to designate the difference
between the two buttons. Once I did that, I put the 16 white means (8 for left arm, 8 for right) in the
backdrop for when you play the sequence. That way, you can see the movement proceed and how close it gets
to each give mean as it goes. I did a screen shot halfway through playing one of the sequences to give
an idea of what it looks like. Hopefully, for next week I'll be able to add the co-variance matrix data
into the mix so that way it'll be better estimated how good each point is doing instead of just judging
it approximately from the dispaly how close it comes to the means.
3/19
Tasks to be Completed:
Last week I made the source code I had been using independently of the PTCons project work with the rest of the project.
This week I had to make the sequence compared independent of the sequences in the point cloud and implement the covariance
matrix.
Progress on Tasks:
The way the program was set up before was a .seq file would be loaded in which would be n sequences.
The first n-1 sequences
would be made into the point cloud, whereas the nth sequence would be the test sequence that would be compared to the point cloud.
One of the things I did this week was to make it so the point cloud would be created and stored, then new sequences could be added
to the n sequences, and the new sequence that is added
could then be compared against the point cloud. You could also select already
made sequences, but that would be trivial since that sequence would be a subset of the pointcloud. I tried to make it so you could
load in a totally new .seq file and use the stored movement to be compared against any given sequence from the new file but that wasn't
possible to implement, because Brandon pointed out to me that whenever a new .seq file is opened a new window is generated within the
program. I also implemented a way to compute a 3x3 covariance matrix for each mean. Here is a sample matrix from one of the
eight means from the left arm.
902.364 1503.1 746.133
1503.1 5139.74 2467.07
746.133 2467.07 1267.29
I will use the covariance matrices generated to help judge movements and to create bubbles around the means.
Possibly, I will use
Open GL to represent a point in the test sequence passing through one of these clouds as the movement is mapped.
I'm going
to talk to Brandon about further information on how to interpret the numbers in the matrices.
3/12
Tasks to be Completed:
Incorporate all the point cloud and mean algorithms in with the rest of the PTCons source files.
Progress on Tasks:
I don't have any fancy diagrams or pretty pictures to show for this week. I've been modifying lots of code and tweaking
it so it will work with the Microsoft compiler. To begin with, I had 2 Matlab source files and 2 C++ source files and
a C++ header file that didn't even work together to achieve the point cloud diagrams and k-means stuff. This past week
I first tried on Unix to get one C++ source file and one C++ header file that would accomplish all the same things
as all the other files I've made in the past month. After merging a bunch of code I got a file that would compile.
I then went to Studio to try to incorporate that source and header file with the PTCons workspace. The Studio compiler is
a little bit pickier on certain things than the g++ compiler was but after some debugging I got it incorporated in with
the rest of the code, and now all my code is called when you hit the "Compute Move" button on the GUI interface that comes
up when PTCons is ran. Tommorrow, before my meeting with Brandon I'm going to see if I can create a text output that
I can plot in Matlab to make sure that it is still accomplishing the same reliable clustering techniques it was before.
With some output to standard out, I was checking on all the variables and mean values and they looked pretty comprable to what
was occurring before so I don't think it will be too much of a problem. There were some slight discrepancies, however, but that
is too be expected because I changed all the int values to doubles to get more accuracy since I would be doing everything in C++
instead of Matlab so speed wasn't as much of an issue.
2/26 & 3/5
SPRING BREAK, made the web site easier to view by changing some fonts & colors and
adding some anchors, and looked at the C++ code to get an idea about how to implement the MATLAB stuff
in Visual Studio but no real tasks were completed.
2/19
Tasks to be completed:
Last week I made some progress with Matlab to be able to print out the point cloud
and then all the points in the comparison sequence, color-coded relating to the distances
away they were from the nearest point in the pointcloud. This could get exhaustive quickly though
with large point clouds and samples to compare it to. Plus, it didn't do much to get rid of noise in
the data or solve the problem of making sure that the movement hit all the areas we wanted. For example,
someone could do a small subset of the range of motion and hit the point cloud dead on, thus yielding a high
score in that area but fail to do the correct motion. With that in mind, this week I was too look into using
a k-mean algorithmic approach to creating clusters within the point cloud to compare the data to. Then, use
a covariance matrix to decide how well the test points fit in with the clusters that represented the point cloud.
Progress on Tasks:
I started out by focusing on making cluster information for either the left arm or right arm. I made the code
flexible enough where by changing a couple variables I could alter between the two, when I clean up the code I'll
probably make it generate data files for both. Anyway, the code follows the k-means algorithm and creates 8 means
for the points to be categorized into. It generates 4 data files for Matlab. Since Matlab recognizes each data
file as a potential matrix, the four files correspond to the x axis of the test data, the y axis of the test data,
the z axis of the test data, and the x,y,z coordinates for the 8 cluster means. After talking to Brandon, I changed
the algorithm to not add a new point to the set of means until all the means had stopped shifting, this made the
means a little more accurate and helped get rid of some data that might get thrown in from random actions and noise.
Figure 1 below shows the after means for the right arm with green circles and the before means with blue circles.

Figure 2 below shows the 8 means for the left arm, this is after making the changes to the algorithm.

After I got all the means stuff taken care of, I went on to compare the test data set against it. The fifth and
final sequence which I have been using as my test set is relatively small, with about 65 points versus the point
cloud which contains about 480 points from the four other sequences. Thus, if you look at Figure 3 for this week,
some of the cluster means have no points that go along with them. The picture is color coded so you can see which
points go with what mean. The points are designated by x's, the means by o's.

I haven't gotten a chance to throw variance into the mix yet, but when I do it will help with finding out
whether points are acceptable within a give distance from a mean.
2/12
Tasks to be completed:
Last week I familiarized myself with the code that outputted
to sequence files. I tried decoding the binary output which
turned out to be much more trouble than it was worth. This
week I was supposed to take a different approach and try
to have the program write ASCII to a file and than import
that into the Matlab program I had previously written.
Progress on Tasks:
I had much more success this week than I did last week.
Brandon helped me initially with some things in Studio
to add a button to the File Menu that I could choose when
I wanted a sequence to be written out. I basically just
cut and paste all the windows operations that had done
the binary archiving to write to a file.
The problem with all this was that Matlab is fairly picky
on the formatting of the data files it excepts so I wrote
a C++ program in Unix to take in the file outputed by
PTCons and have it in turn output a series of data files
that correspond with the matrices needed by Matlab to make
the data. I then made some modifications the the Matlab file
I created so that it would take in the X, Y, and Z points
from the first n-1 sequences in a file and then compare
that point cloud with the nth sequence. The resulting printout
is stapled to this printout. The *'s designate members of the
pointcloud and the x's designate the last sequence that is being
compared to the pointcloud. The picture I included on the web
page makes it much easier to see the x's since they are different
colors than the point cloud (red or blue, depending on the distance
away). Right now there are a few variables hardcoaded in that
I could make command line so other sequences can be looked at
but other than that the program works pretty good at analyzing
the data.
Plots generated by Matlab for the comparison of Sequence 5 with 1-4 from
the file "PTCons5.seq."

2/5
Tasks to be completed:
Last week I worked on representing the point cloud in Matlab
but I used artifical data to test it. This week I was
supposed to find out a way to use the data from the sequence
files that were created by PTCons and put them in a form
that Matlab could use. This would involve familiarizing
myself with the sequence storage functions in the code
and figuring out how to do binary input and output.
Progress on Tasks:
I spend a few hours going through all the calls and
overloaded operators relationg to the streams that
performed storage and it basically came down to something like this.
I'll put << in front of something
to designate that it is outputed to the sequence file.
.seq
<< filenum_seqs (unsigned int)
for (0:num_seqs){
(from PTSeq32.cpp)
<< ready_to_play (BOOL)
<< height_cm (float)
<< combo_index (int)
<< seq_num (int)
(from pt_raw_sequence.cpp)
<< num_snaps_taken (unsigned int)
for (0:num_snaps_taken){
head_snaps
(from pt_snapshot)
<< x_coord (double)
<< y_coord (double)
<< z_coord (double)
<< t_stamp (unsigned long)
<< t_coord (double)
left_hand
(from pt_snapshot)
<< SAME AS HEAD_SNAPS
right_hand
(from pt_snapshot)
<< SAME AS HEAD_SNAPS
sternum
(from pt_snapshot)
<< SAME AS HEAD_SNAPS
}
(from pt_norm_sequence.cpp)
TONS OF NORMALIZED OUTPUT
}
I had some problems with the binary input.
I tried multiple functions in C/C++, basically just
looking to see if I could at least
read in the first number from the sequence file
which should have been 4.
In the end I thought I had a textbook C and C++ file,
that used their respective IO library files and should
have done the same thing and both gave me not only
wrong answers, but different ones. I might have been
wrong with my implementation which is why I'm including
both functions here and the answer that was printed.
C++
void main(){
unsigned int price;
ifstream seq("PTCons5.seq",ios::binary);
seq.read(&price,4);
cout<
seq.close();
}
Answer: 4290704556
C
int main()
{
unsigned int num_seqs;
int c;
FILE * f = fopen("PTCons5.seq","rb");
c = fread(&num_seqs,4,1,f);
printf("%d\n\n",num_seqs);
fclose(f);
}
Answer: 83886080
I also was unsure of the little endian-big endian
thing so I took Brandon's advice and tried the
ntohl, ntohs, htonl, and htons functions to switch byte order but
in both cases they just printed the same thing. Since
it printed the same thing, I thought it might still be
the byte order so I downloaded a binary calculator and
entered both numbers in and switched the byte order and
it still didn't come up with anywhere near a number
that would tell me I was on the right track. That's
where I'm at progress wise for reading in the binary
file. Hopefully, after talking with Brandon on Monday
I'll be able to see where I went wrong in that department
and also verify what I interpreted to be the flow of
variables to output in the program.
1/29
Tasks to be completed:
Read the paper "Continuous Dynamic Time Warping for translation-invariant curve alignment with applications
to signature verification" by Munich and Perona. I also needed to try to use Matlab to implement a point cloud
where you could easily check a sequence against the point cloud to see how close it came to the cloud.
Progress on Tasks:
I read the paper, but most of my time this week was spent doing the Matlab stuff. I spent the early part of the
week refamiliarizing myself with the general features of Matlab and the plotting functions. Actually, a big
problem I ran into ( i ended up finding the answer in a obscure Matlab bible in the lab) was trying to map a series
of points on a graph without a)having the points connected by a line and b) having all the points appear on the same
graph. The default in Matlab is to plot a point, whether 2-D or 3-D, and then to plot the next point in a new graph.
I ended up getting the kinks out and created 3 4x100 matrixes: X, Y, and Z (one for each axix). The first three rows
in each dimensional matrix was supposed to represent the point cloud and the last one would represent the sample
we were throwing at the point cloud. I then had magneta *'s on the 3-D graph represent the cloud (only one row in X,Y,Z
for this demo instead of 3) and different colored x's for the sample dependent on how far those points are away
from the closest representative of the cloud. The points used for the cloud were
X: (absolute value of sin(x))*100, 1:x:100
Y: (absolute value of cos(x))*100, 1:x:100
Z: x, 1:x:50; 50-x, 50:x:100
These were the best formulas I could think of to make the cloud a somewhat consistent series but have the points fairly
spread out at the same time. The test set was just a randomly generated 3X100 matrix.
Here is the resulting pointcloud and comparative set when ran through Matlab:

1/22
Tasks to be completed:
Look into ways to use dynamic programming to assist in the problem of comparing two sets of
motions as a function of time against one another.
Progress on Tasks: I read Chapter 16 in "Introduction to Algorithms" by Cormen,
Leiserson, and Rivest dealing with Dynamic Programming. The problems I ran into is that to
create the subproblems to be used to find the ultimate comparison we must first figure
out a kind of fitness function to use for these sets of points. From there, I was thinking
we could divide all the points into an arbitrary number of segments. However, instead of
mapping the points from the two sets of data to each other from segment to segment in a one
to one fashion I think it would be best to use a dynamic programming approach of doing
some kind of fitness function of maybe 1,2,3, etc. segments from one set of data points
to maybe only one segment of another. With that, you could keep combining sets of data
until at last you have it figured out which subsets of segments in relation to subsets of
data from the other group give you your best solution. I'll discuss this with Brandon and
then see where to go from there.
1/15
Tasks to be completed:
Given papers concerning Speech Recognition (by Sanders) and Human Body Tracking in a paper by
Christopher Wren, Ali Azarbayejani, Trevor Darrell, and Alex Pentland entitled "Pfinder:
Real-Time Tracking of the Human Body."
Progress on Tasks: Read the papers and discussed them with Brandon.
|