Tutorial 2: Linear Dynamical Systems (scrolly)

Tutorial objectives¶

Estimated timing of tutorial: 1 hour

In this tutorial, we will be learning about behavior of dynamical systems -- systems that evolve in time -- where the rules by which they evolve in time are described precisely by a differential equation.

Differential equations express the rate of change of the state variable $x$, written as the derivative $\dot{x} = dx/dt$:

Today, the focus is on linear dynamics, where $f(x)$ is a linear function of $x$.

Tutorial slides¶

Here are the slides for the whole tutorial. You can also download them.

Section 1: One-dimensional differential equations¶

This video introduces dynamical systems as the mathematics of things that change in time, including timescales relevant for neuroscience. It covers the definition of a linear system and walks through solutions to one-dimensional, deterministic dynamical systems, their behaviors, and stability criteria.

This section is a recap of Tutorials 2 and 3 of the pre-course calculus day.

The exponential solution¶

A one-dimensional differential equation in $x$ of the form

where $a$ is a scalar, has solutions

where $x_0$ is the initial condition: the value of $x$ at time 0. The sign of $a$ decides whether $x$ decays, stays put, or grows.

Forward Euler integration¶

We can simulate an ODE by treating time as a list of steps $t_0, t_1, t_2, \dots$ with $t_{i+1}=t_i+dt$. From the definition of the derivative, the small change over $dt$ is

So at each step we add that small change to the previous value:

This scheme, forward Euler integration, works well when $dt$ is small and the equation is simple. It struggles with noisy dynamics or sudden big jumps in $x$ (as in models of excitable neurons). For our system it works just fine.

Implement forward Euler¶

Referred to as Exercise 1B in video

Complete integrate_exponential to compute the solution of $\dot{x} = a x$ with forward Euler integration, then run it to plot the solution over time.

If you skip ahead, the cells after this one use the reference solution, so they still work.

Euler against the exact solution¶

With a small step, forward Euler sits on top of the exact solution $x_0 e^{at}$. With bigger steps, it becomes choppy and drifts away from it.

Forward Euler integration¶

1. What happens when you change $a$? Try values where $a<0$ and $a>0$.

2. $dt$ is the step size of the forward Euler integration. Try $a = -1.5$ and increase $dt$. What happens to the numerical solution when you increase $dt$?

Section 2: Oscillatory dynamics¶

Estimated timing to here from start of tutorial: 20 min

We will now explore what happens when $a$ is a complex number and has a non-zero imaginary component.

Oscillatory dynamics¶

Referred to as exercise 1B in video

Change the real and imaginary parts of $a$ (so $a$ = real + imaginary $i$).

1. What values of $a$ produce dynamics that both oscillate and grow?

2. What value of $a$ is needed to produce a stable oscillation of 0.5 Hertz (cycles/time units)?

Section 3: Deterministic linear dynamics in two dimensions¶

Estimated timing to here from start of tutorial: 33 min

This video introduces two-dimensional, deterministic dynamical systems written as a vector-matrix equation. It covers stream plots and how to connect phase portraits with the eigenvalues and eigenvectors of the transition matrix A.

Two variables¶

Adding a second variable (or dimension) adds more variety of behaviors, for example in systems of several neurons. Two variables in isolation evolve as

To make things interesting, we add interaction terms ${a}_{12} x_2$ and ${a}_{21} x_1$. The two equations then become one vector-valued linear ODE, where $\mathbf{x}$ has two elements and $\mathbf{A}$ is a $2 \times 2$ matrix.

Sample trajectories in 2 dimensions¶

Referred to in video as step 1 and 2 of exercise 1C

We simulate trajectories of the system

and plot how $x_1$ and $x_2$ evolve in time. An integrator from scipy does the solving, and the helper plot_trajectory draws the result. Write the system function for a linear system with two variables.

Varying A¶

Pick different options for $\mathbf{A}$. What kinds of qualitatively different dynamics do you observe?

Hint: Keep an eye on the x-axis and y-axis!

Varying initial conditions¶

Now vary the initial conditions for a fixed $\mathbf{A}$:

What kinds of qualitatively different dynamics do you observe? Hint: Keep an eye on the x-axis and y-axis!

Section 4: Stream plots¶

Estimated timing to here from start of tutorial: 45 min

Plotting trajectories one initial condition at a time is tedious. A stream plot shows how a whole grid of initial conditions evolves.

Think of an initial condition ${\bf x}_0=(x_{1_0},x_{2_0})$ as a position in a space. At each position $\bf x$, a stream plot draws a small arrow for $\bf Ax$ and connects the arrows into stream lines. Since $\dot {\bf x} = \bf Ax$, the stream line through ${\bf x}_0$ shows ${\bf x}(t)$.

Interpreting eigenvalues and eigenvectors¶

Here are the stream plots for each option of A from Demo 3A, with the eigenvectors of $\bf A$ as a red line (1st eigenvalue) and a blue line (2nd eigenvalue).

What is special about the direction in which the principal eigenvector points? And how does the stability of the system relate to the corresponding eigenvalues? (Hint: from your introduction to linear algebra, for matrices with real eigenvalues, the eigenvectors are the lines on which $\bf Ax$ is parallel to $\bf x$, and the eigenvalues are the factors by which $\bf Ax$ is stretched or shrunk compared to $\bf x$.)