Friday, May 26, 2017

Applications: A list of projects using matrix operations

Matrix Operations

I've just finished teaching Linear Algebra twice since the beginning of the year, and I'll be teaching it again in the fall. It is time that I cleaned up my applications list and updated the project files. Since I am enumerating them, I might as well do it here. Here is part 1 on matrix operations.  Others may be added later.

Seriation in archaeology - Uses incident matrices, matrix multiplication and transposition, and the properties of symmetric matrices to determine relative ages of sites with common artifacts.  I haven't used this in my classes yet, but there seem to be some good resources available including "Some problems and method in statistical archaeology," David Kendall, World Archaeology, 1969, which is available in JStor.  It also appears in Gareth Williams Linear Algebra with Applications.

Color manipulation in images - Uses matrix multiplication to alter colors in the RGB scale.  This article by Paul Haeberli describes the 4x4 matrices needed to modify colors, including offsets.  I haven't used this in class yet, but I could see this as a good project to have students work in Mathematica.  It also is a companion for transformations in 3D graphics, which also use 4x4 matrices.  The ability to use matrix multiplication to add vectors is common to both areas.

Image color conversion - Uses matrix multiplication to convert from, say, RGB to YIQ, color models.  I haven't used this in class, but it shows up in Gareth Williams Linear Algebra with Applications. This article by Ford and Roberts describes a bevy of color models, and it seems that only some conversions are linear.  Without a way to test whether the color conversion is correct, I don't see this as an interesting project. However, maybe Mathematica can render the other color models.

Transformations in 2D graphics - Uses matrix multiplication to apply rigid and non-rigid transformations to images.  May or may not use projective coordinates, depending on whether translations are allowed.  Resources abound.

Projection of 3D images onto the plane - Uses matrix multiplication to project 3D images onto the plane given the coordinates of the image and the location of the viewer.  Uses projective coordinates.  I use a paper written by Jeanie Mullen, one of my honors students.  This project has worked best for students with programming backgrounds.

Two-port in an electrical circuit - Uses matrix multiplication to describe the change in voltage and current through a two-port or a series of two-ports.  A simple application of Ohm's law that creates two linear equations that can be described using matrix multiplication.  The equations relate the input current and voltage to the output current and voltage.  This Wikipedia article has a table of many transmission matrices and their effect.

Wednesday, May 24, 2017

Applications: A list of projects using eigenthings

Eigenthings

Gould's accessibility index in a network - The process uses a modified adjacency matrix and the components of the eigenvector associated with the dominant eigenvalue. Students find this approachable and adaptable.  Applications to historical geography, air traffic.

Discrete dynamical systems - Using linear algebra to study discrete dynamical systems comes in several flavors.  Here are some projects that students find interesting and that differ from each other enough that they feel they are not repeating someone else's project.


  • Difference equations and the Fibonacci sequence - Using eigenvalues to write the product of the nth power of a diagonalizable matrix and an initial vector allows one to write a closed form for a recursive formula.  Matrices of size 2x2 are needed to write the closed form of the nth Fibonacci number, but students can easily move from there to the closed form of 3rd and 4th order difference equations.  This project is always chosen by some student even though it is not applied to a real-world situation.

Monday, May 22, 2017

Applications: A list of projects using matrix inverses

Matrix Inverses


I separate these projects from those other using matrix operations, because I make a clear distinction between forward and inverse problems in my classes.

Cryptography - These projects come in two varieties: using modular arithmetic and not.

  • Matrices with |determinant| = 1 - Uses matrix multiplication to encode a matrix and multiplication of the inverse to decode.  Any matrix with determinant 1 or -1 will result in an inverse with integer components.  Students tend to be drawn to these projects, but sometimes I find it hard to push them further, such as requiring them to create their own encoding matrices, etc.  Resources abound.
  • Modular arithmetic and row reduction - Uses matrix multiplication in modular arithmetic to encode and decode a message and row reduction in modular arithmetic to find the inverse.  This is not that the decoded matrix is read using mod 26, but rather that the matrix operations are done with, say, mod 37.  This project requires a little more tenacity on the part of the student, and this article by Keith Conrad discusses inverses of matrices under modular arithmetic.





Saturday, July 31, 2010

Interesting property of the inverses of Magic Squares

I’m a nerd. I freely admit it. Sometimes I see a result in mathematics that I think is fun, even if it is not immediately useful. I intended for this blog to be about applications of linear algebra, and by that I mean useful, if not a little esoteric, applications; applications that are juicy and interesting and have good visuals. Although the abstract applications of linear algebra to theoretical areas of mathematics are useful to someone, they do not have the hands-on feeling that I want. However, the nerd in me finds some abstract ideas sexy enough to include in this blog. The following is one of them. Although there is a connection between this idea and Magic Squares, which have constant row sums, I don’t see an application right off. If you know of one, please let me know.

Theorem (already this post looks different than usual ‘cause it has a theorem):
If an invertible matrix A had constant row sums of k, then the inverse of A has constant row sums of 1/k.

Proof (Oh, no.  A proof.  Just when this blog looked like it was just fun stuff):
Let A be an m-by-m invertible matrix with constant row sums of k. Let B be the inverse of A.  Now, AB = I and the diagonal elements of I are all 1.  Note that I has a constant row sum of 1.  Hmmm.  Let

             (1)
be the elements of I.  Then
                  (2)
is the sum of the elements of row i of I, and
              (3)
is the sum of the elements of row n of A.  Now, write the elements of a row of I = BA as the sum of products of elements of A and B:
             (4)
Now, we're ready to put this all together.
Thus,

         (5)

but the right-hand side of (5) is the sum of row i of B

It seems like a trick, and in a way it is, but the trick is legit.  What we have done is started with the sum of row i of I, written it in terms of the elements of A and B, and then seen that we could factor out the elements of B leaving row sums of A.  To help you understand this, carefully write the elements of a general 3-by-3 BA in terms of the elements of B and A.  Now, sum one of the rows of BA and rearrange so you can factor out the various elements of B.  The sums of the elements of A left will each be a row sum.  This wouldn't be a proof in general, but it should help you understand what is happening in that sum-switching step of the proof, and that it works because the elements of a matrix product are sums and then we sum a row of sums, and the terms within these two sums can be conveniently rearranged.

Questions

  1. We started with an invertible A with constant row sum.  What if A had constant column sum of k instead of row sum?  Will the inverse of A have constant column sum of 1/k as well?  How would the proof go for that?  This would be a great exercise in working with sums and indices.

  2. This proof will not work if the constant row sum is k = 0.  Can a matrix have an inverse if k = 0?   If k is not zero, are we guaranteed that A is invertible?  Can we determine if a magic square is invertible by the row sum alone?

  3. Is there a relationship between the diagonal sums of a magic square and its inverse?

  4. Is the inverse of a magic square a magic square?  Is the inverse of a semi-magic square a semi-magic square?

  5. Is the square of a magic square a magic square?  Is the square of a semi-magic square a semi-magic square?  Cubes?  Fourth powers?
Reference:  Wilansky, Albert, "The row-sums of the inverse matrix,"  The American Mathematical Monthly, Vol. 58, No. 9. (Nov., 1951), pp. 614-615.

Friday, July 30, 2010

Euler Characteristic and Planar Graphs

To the right we see a planar graph. It divides the plane into 5 regions which I have labeled A, B, C, D and E. We call region D a triangle because it has three sides. Regions B, C and D are quadrilaterals since they have 4 sides. Even though A is an infinite region in the plane, it has 3 sides and is called a triangle. The question we address here is whether we can draw planar graphs with all possible combinations of triangles, squares, pentagons, etc., or not. Also, can we determine which combinations are possible?

We will use the Euler Characteristic for the sphere to solve this problem. I know it looks like the figure above is drawn on the plane, but you can also think of it as drawn on a relatively flat part of a sphere. In this way, region A is not infinite, but we will still call the regions triangles, quadrilaterals, pentagons, etc., even though they now have a curve to them. The Euler Characteristic is VE + R, where V = number of vertices, E = number of edges (not the region E above) and R = number of regions. The Euler Characteristic depends only on the surface on which the planar graph is drawn, and not the shape of the graph. For the sphere,

VE + R = 2,

always. In the example at the above, VE + R = 6 – 9 + 5 = 2.

Let’s do a little counting. In the figure above, E = 9 is the number of edges. However, if we count the sides of the polygons we get 2 triangles x 3 sides each + 3 quadrilaterals x 4 sides each = 18 sides. This is twice as many as the edges, because polygon each side is counted twice for each edge, once for the polygon on one side of the edge and once for the polygon on the other side of the edge. For instance, in the edge count, E, the edge xy is counted for the triangle D and the quadrilateral C.

What about the vertices? In the figure above V = 6is the number of vertices. If we count the vertices of the polygons we get 2 triangles x 3 vertices each + 3 quadrilaterals x 4 vertices each = 18. In this case, we get three times as many polygon vertices as graph vertices because there are three polygons meeting at each vertex. For example, polygons A, B and C meet at vertex t.

These counting techniques and the Euler Characteristic will give us a system of equations for finding whether graphs with certain combinations of polygons are possible. 

Example 1: Can we draw a planar graph with only triangles so that exactly three triangles meet at each vertex? If so, how many triangles will there be? We can answer this question with a system of linear equations. The first equation is the Euler Characteristic for the sphere:

VE + R = 2.

Each region is 3 sided, but if we count 3R sides that will be double the edges since each side is counted twice:

3R = 2E.

Each region has 3 vertices, but if we count 3R vertices that will be triple the total vertices since three triangles meet at each vertex:

3R = 3V.

Solve this square system of 3 equations in 3 unknowns using your favorite method, and we find there is only one way to do this:

V = 4, E = 6 and R = 4.

The only solution is to have 4 triangles, 4 vertices and 6 edges as shown on the right, remembering that the outside region is a triangle.  So, we could never draw a graph that had 5 triangles such that 3 triangles meet at each vertex.  Give it a try to see why it can't be done.

Questions: Can we draw a planar graph of triangles where 4 triangles meet at each vertex? 5 triangles meet at each vertex? 6 triangles meet at each vertex? How would we change the system above to answer these questions.  If the graph exists, try to draw it. 

Example 2: What if there are two different types of polygons? Consider a graph of triangles and quadrilaterals, assuming that three polygons meet at each vertex. We will introduce two new variables: T and Q, the counts of the triangles and quadrilaterals, respectively. Now, the total number of regions is the sum of the two types of polygons,

T + Q = R.

Count the edges, 3 for each triangle and 4 for each quadrilateral, and as in Example 1, this counts each edge twice:

3T + 4Q = 2E.

Count the vertices, 3 for each triangle and 4 for each quadrilateral, and as in Example 1, this counts each vertex thrice, because 3 polygons meet at each vertex:

3T + 4Q = 3V.

Finally, we need the Euler Characteristic:

VE + R = 2.

This time we’ll use a matrix and row reduction to get the solution. The system is underdetermined, so we expect to get infinitely many solutions.



Sure enough, we have a free variable and we can write the general solution as

T = 12 – 2R,
Q = –12 + 3R,
V = –4 + 2R and
E = –6 + 3R.

But in this application, the values of T, Q, V and E have physical meaning and must be positive. If V or E is zero, then the graph would be empty. We could assume that T or Q is zero, but we are interested in graphs with both triangles and squares. Now we can solve the inequalities below to see if there is a viable solution, and how many there are.

T = 12 – 2R > 0      =>      R < 6
Q = –12 + 3R > 0      =>      R > 2
V = –4 + 2R > 0      =>      R > 2
E = –6 + 3R > 0      =>      R > 4

Okay, R is an integer and strictly between 4 < R < 6, so R = 5 is the only realistic solution to this underdetermined system. Now,

R = 5, T = 2, Q = 3, V = 6 and E = 9.

Draw this graph (don’t forget that the outside region is one of the 5 regions and must be either a quadrilateral or a triangle). The graph is at the bottom of this blog, but don’t peak before you give it a try.

Questions:

1. Can you draw a planar graph with pentagons and hexagons such that three polygons meet at each vertex? If so, how many of each polygon are there? Can you draw them?

2. Can you draw a planar graph with triangles and quadrilaterals such that four polygons meet at each vertex? I have written the equations and solved the system for this case, and this may have infinitely many solutions, but I haven’t had the time to draw more than two of the solutions and would like to see an algorithm for drawing all of them.

3. Other surfaces, such as a torus (donut) have different Euler Characteristics. How does one draw a graph on a torus? What are the solutions to the questions above if the graphs live on a torus?  Wolfram MathWorld has a list of the Euler Characteristics for surfaces, but WikiPedia has nice images of those surfaces if you scroll to the bottom of the article.

To limit this blog to a few pages, a lot is left unsaid. But again, these posts aren’t meant to give an in-depth discussion of the topic, but just an introduction.  Go exploring for more about this topic.

Reference: Alain M. Robert, An Approach of Linear Algebra through Examples and Applications

Thursday, July 22, 2010

Solving Balance Puzzles with Matrices

I grew up on puzzles and games. A typical Christmas or birthday present was a puzzle book, and our family spent a lot of quality time together when the Games magazine came in the mail. Being the math nerd that I am, I would sometimes use trial and error for the puzzles, but anything I could convert into equations to solve was a bonus. So, in a way, I was introduced to linear algebra long before I took the class in undergraduate school.

These puzzles are from Games magazine, and I hope they forgive me for forgetting the specific issue for a complete citation. They provide a nice visual from which systems of equations can be developed. These systems tend to be underdetermined; however, there are added restrictions on the solutions--they must be integers within a certain range. Thus, these underdetermined systems have a finite number of solutions instead of infinite, specifically because equations are over a range of integers, not over the reals.

Think of these puzzles like building a mobile, where the objects on each side must balance so that the horizontal bars stay horizontal under the following rules.
  • each shape represents a unique positive weight
  • each weight is an integer N such that 0 < N < 100
  • the right and left sides of each horizontal beam must balance
  • a piece hanging directly below the fulcrum does not affect the balance between the left and right arms
  • the size of the pieces has no relation to weight
  • these are exercises in balancing number values and do not take into account the distance from the fulcrum
Example 1: For the mobile at the top of the page, the total weight is 36. I have labelled each weight with a variable name, A, B and C. The total weight is 36:

A + 2B + 2C = 36.

The left arm must balance the right arm:

A + 2B = 2C or A + 2B - 2C = 0.

We create an augmented matrix for this system and row reduce:


While using a matrix was not necessary for this simple problem, it will be useful for the larger systems below. So, C = 9, B is a free variable, and the general solution is

(A, B, C) = (18 - 2B, B, 9)

with some restrictions on B. Since 0 < A < 100,

0 < 18 - 2B < 100,
-18 < -2B < 82, and
9 > B > -41.

Since B > 0, we find 0 < B < 9, and thus, there are 8 possible answers, one each for B = 1, 2, 3, 4, 5, 6, 7 and 8.

Questions: First, solve the puzzles below, then consider these questions.
  1. What would it take to create one of these puzzles? Can you create a puzzle that has a unique solution?
  2. How would it change if we did consider distance from the fulcrum? Would it still be a linear system? If so, create a simple puzzle where distance is considered.
  3. What determines how many equations are required? For instance, in Puzzle 1 below how many equations are described?  Don't forget that each horizontal bar needs to balance, not just the top one.
Puzzle 1. The total weight is 80. Only one shape weighs more than 9 and the cube is 1 unit heavier than the octahedron.

Puzzle 2. The total weight is 54. The three arms are equal in weight.

Puzzle 3. The total weight is 180. The three arms are equal in weight.
---------------------------------------------
A later post - December 31, 2011

I found a link with more balance puzzles.  These are different looking, but the idea is the same.  In this case you need to take the distance each weight is offset from the balance point into account.  For example, a weight of 2 units that is 1 unit from the fulcrum will balance a weight of 1 unit that is 2 units from the fulcrum.  Can you use linear algebra to solve these puzzles?  If so, how?

Here is another page of balance puzzles.http://battleships.free.fr/balance_i.pdf

Wednesday, July 21, 2010

Gould Index - Matrix Application to Geography

Figure 1:
Map of 7 towns connected by 9 roads
The Gould Index has a nice visual that students enjoy, but it also is used to answer a question (important for applications) and can lead to discussions of the PageRank function used by Google.  Much of this post is adapted from a paper by Patrick Carlson, one of my linear algebra students.  He was interested in networks, and did a great job of understanding and explaining the general nature of the mathematics behind the Gould Index.

Example 1:  Consider a graph that respresents a set of towns (the vertices) and the travel routes between those towns (the edges) like in Figure 1.  Historical geographers were interested in which town would become the trade center for this region.  Now, you are saying to yourself that this is obvious in this picture, but let's see if some mathematics can come to the same result we do.  If so, then let's apply it to a less obvious graph.

Figure 2:  Adjacency matrix for Figure 1
First, make the adjacency matrix for the graph (Figure 2).  Place a 1 in position (U, V) when U and V have a travel route between them and a 0 in the (W, T) position when there is no direct travel route between W and T.  We will also place a 1 in each diagonal position, such as (Q, Q) as if there was a loop there, since clearly if you are in Town Q you can obviously get to Town Q.  Notice that this matrix is symmetric, which will make some of the mathematics work but is also a way of checking to make sure you created the matrix correctly.

Now, here's the math.  Find the eigenvalues of this matrix and find the largest one in absolute value.  We get eigenvalues (2, 0, 4, -1, 0, 2, 0).  The third one, 4, has the largest absolute value.  Now, find the eigenvector associated with the eigenvalue of 4: 
(0.3162, 0.3162, 0.3162, 0.3162, 0.6325, 0.3162, 0.3162).  Normalize this vector by dividing by the sum of the entries, 2.5297, to get

(Q, R, S, T, U, V, W) =
(0.125, 0.125, 0.125, 0.125, 0.25, 1.25, 1.25).

These are the Gould indices of each of the vertices, which describes how strongly each vertex is connected to the other vertices.  As we knew, U is the most strongly connected with a Gould Index of 0.25, and the others are equally connected because of the symmetry of the graph.

Figure 3:  Another graph
Example 2:  What if we looked at a graph that more realistically represents a set of towns, which may have rivers, mountains or other geographical features preventing travel routes between towns, as in the graph in Figure 3.  In this case, would G be the trade center because it is directly connected to the most towns, or would it be C because between C and every other town is at most one town?  Create the adjacency matrix (with ones along the diagonal), and find the eigenvalues which are
(4.01, -1.37, 1.71, 1, -0.37, -0.56, 2.58, 1, 1).  The first is the largest and the eigenvector associated with it is
(0.2941, 0.4097, 0.5023, 0.3249, 0.3026, 0.1359, 0.4770, 0.1583, 0.1583) and the sum of the entries is 2.7631.  Normalize by dividing by the sum of the entries to get the Gould indices:

(A, B, C, D, E, F, G, H, I) =
(0.1064, 0.1482, 0.1818, 0.1176, 0.1095, 0.0492, 0.1726, 0.0573, 0.0573).

Of course, we can see which Gould index will be largest before normalizing, but there are other applications for which normalized values of the eigenvector will have meaning, so it doesn't hurt to get used to normalizing.  We see that C has the largest Gould Index with 0.18, but G comes in a close second with 0.17.  In this calculation, more value is given to vertices that are closer to others along paths of length more than 1 than to vertices that connect with a path of length 1 to more vertices.  Note that I and H have the same smallest index, as expected.

What is going on?  Although we have some empirical evidence that the Gould index does what is expected, what is the theory behind it?  A version of the Perron-Frebonius Theorem says that if a matrix is nonnegative and primitive, then there will be a real eigenvector that dominates the others in absolute value, it will have multiplicity 1, and the eigenvector associated with it will be real and positive [C].  The adjacency matrices above are nonnegative since all the entries are 0 or 1, and for the two above I checked that they are primitive; in Example 1, the square of the matrix was positive, and in Example 2, it took the fourth power of the matrix before getting a positive matrix.  We don't use the adjacency A, per se, but instead we use B = A + I when we put 1s in the diagonal.  Try a bipartite graph and see that the powers of A will never be positive, and thus A is not primitive in that case, but B = A + I will be.

Straffin discusses the how of the Gould index in more detail, and discusses its connection to finding the dominant team in a round robin tournament and measuring the importance in nodes in a communications network.  He has a significant list of references to examples of this method along with references to other uses of linear algebra and graphs in geography.  This method is a beginning for the mathematics behind the algorithm used in PageRank used by Google; however, the adjacency matrix (with 1s on the diagonal) is not primitive and is too large to find the eigenvalues and eigenvectors directly.  But that is a discussion for another day.

Questions:  When this model is applied to historical travel routes, the trade center predicted by the model does not always match the trade center that develops.  Non-geographical issues may be involved here, such as politics and wealth, but it is clear that the model does not take into account the time of travel.  Maybe there is a train between two towns for quick access, but two others are only accessible on foot through a mountain pass.  Can we change the results for Example 2 above by considering a weighted graph instead?  If so, should we weight with time of travel or the reciprocal of time of travel?  How would the diagonal elements be weighted?

The nth power of an adjacency graph gives the number of paths of length n between vertices x and y in the (x, y) position.  Could this be used in someway to predict the trade center?

In Gould's article is a study of the travel routes of Uganda in 1921 and 1935, and Straffin's paper has references to other studies.  Can you reproduce these results?

References:

[C] Hal Caswell, Matrix Popoulation Models, Sinauer, 1989.

[G] P. R. Gould, "On the geographical interpretation of eigenvectors,"  Transactions of the Institute of British Geographers, No. 42 (Dec., 1967), pp. 53-86.

[S] Phillip D. Straffin, "Linear Algebra in Geography: Eigenvectors of Networks," Mathematics Magazine, Vol. 53, No. 5 (Nov., 1980), pp. 269-276.