Scientific Computing - Jim's Numeric Work

About

A workspace for

* Trying out ipython (which is very cool, by the way).
* Explaining and coding the topics that Alex and I have been discussing.

Jim Mahoney | cs.marlboro.edu | Oct 2013 | MIT License

Theory

second order numerical differential equation

There are several approaches (including particularly the 4th order Runge-Kutta) but here I'll just do the simplest thing. If \(y\) is the dependent variable and \(x\) is the independent, put \(x\) on a grid with spacing \(dx = x[n] - x[n-1]\) where \(n\) is a discrete integer index labeling the grid points.

Then the first derivative is approximately

\[ y'(n + 1/2) = (y(n+1) - y(n))/dx \]

and likewise the second is (to \(O(dx^2)\))

\[ y''(n) = (y(n + 1/2) - y(n - 1/2))/dx \].

Plugging in gives

\[ y''(x_n) = \frac{y(x_{n+1}) - 2 y(x_n) + y(x_{n-1})}{dx^2} \]

or

\[ y_{n+1} = 2 y_n - y_{n-1} + dx^2 y''_n \]

which is implemented in diffeq2

harmonic oscillator

This 2nd order differential equation

\[ \frac{d^2 y}{dx^2} = - y \]

has exact solutions

\[ y = A sin(x) + B cos(x) \]

'Nuff said.

exact pendulum

So let's look at a problem that doesn't have an exact solution.

The equation of the exact pendulum is

\[ m \frac{d^2 L \theta}{dt^2} = - m g \sin(\theta) \]

or

\[ \frac{d^2 \theta}{dt^2} = - \frac{g}{L} \sin(\theta) \]

where \(m\) the mass, \(L\) and \(g\) are constants, while the angle \(\theta\) and time \(t\) are the dynamical variables. (The force is negative because it is in the direction of decreasing \(\theta\).)

Defining dimensionless variables \(x\) and \(y\) in terms of a time scale \(T\) gives

\[ t = x T \] \[ y = \theta \] \[ \frac{d^2 y}{dx^2} = - (T^2 \frac{g}{L}) \sin(y) \]

Choosing \[ T = \sqrt{\frac{g}{L}} \] leaves just

\[ \frac{d^2 y}{dx^2} = - \sin(y) \]

which is nearly the harmonic oscillator but with a nonlinear \(sin(y)\) term. For large amplitudes, the solution won't look like \(sin(x)\).

quantum one dimensional stationary states via "shooting"

The well-known 1D time independent Schrodinger equation is

\[ - \frac{\hbar^2}{2 m} \frac{d^2 \psi_n(x)}{dx^2} + V(x) \psi_n(x) = E_n \psi_n(x) \]

where \(V(x)\) is a binding potential energy, \(\psi_n\) and \(E_n\) are the \(n^{th}\) bound states.

Let's first do the quantum harmonic oscillator, which has the potential

\[ V(x) = \frac{1}{2} k x^2 \]

The first step before we can work this numerically is to change it to a dimensionless form. Define a distance scale \(X\), and energy scale \(\varepsilon\), and dimensionless variables \(s, C_n, y_n\) :

\[ s \equiv \frac{x}{X} \] \[ C_n \equiv \frac{E_j}{\varepsilon} \] \[ y_n \equiv \Psi_n \sqrt{X} \]

(The units on \(\Psi\) can be a bit confusing. Just remember that \(\Psi^2 dx\) is a probability, which is dimensionless.)

Defining two energy constants

\[ Kinetic = \frac{\hbar^2}{2 m X^2} \] \[ Potential = \frac{1}{2} k X^2 \]

lets us write the dimensionless equation as

\[ \frac{d^2 y_n(s)}{ds^2} = \left( \frac{Potential}{Kinetic} s^2 - \frac{\varepsilon}{Kinetic} C_n \right) y_n(s) \]

Since we have two scale factors \(X\) and \(\varepsilon\) which we can set, both of these energy ratios can be set to 1. In other words, the following two equations define the constants \(X\) and \(\varepsilon\) in terms of \(m, \hbar, k\) :

\[ \varepsilon = \frac{\hbar^2}{2 m X^2} \] \[ \frac{1}{2} k X^2 = \frac{\hbar^2}{2 m X^2} \]

and we want to solve this for \(y(s)\) :

\[ \frac{d^2 y_n(s)}{ds^2} = \left( s^2 - C_n \right) y_n(s) \]

for the quantum numbers \(n = (0,1,2,3,...)\).

discrete fourier transform

Given \(N\) points \(x_n\) where \(0 \le n \le N-1\), the complex DFT is defined as

\[ X_k \equiv \sum_{n=0}^{n=N-1} x_n \omega^{- k n} \]

where \(\omega = e^{(0+1j) 2\pi / N}\) is the \(n^{th}\) complex root of 1. (I'm using Pythons notation for complex numbers here, with \(\sqrt{-1}=(0+1j)\).

The inverse transform is

\[ x_n = \frac{1}{n} \sum_{k=0}^{k=N-1} x_n \omega^{ k n} \]

TODO :

Talk about linear algebra, quantum bras and kets, and all that?

Discuss FFT?

Code

See http://ipython.org/ for installation and docs. Start ipython from the command line with $ cd ~/academics/term/fall2013/scientific_computing/jims_ipython_numeric_work $ ipython notebook --pylab=inline which launches the browser. Click on jims_numeric_work to visit this notebook. To create a static .html output file : $ ipython nbconvert --to html jims_numeric_work.ipynb
In [70]:
from cmath import exp    # complex exponential, i.e. exp( 0+1j )
import numpy

def diffeq2(d2y_dx2, x0=0.0, dx=0.01, N=100, y0=0.0, y1=0.01):
    """ Return y[], a numerical solution to 
        the 2nd order differential equation d2y_dx2(x,y) = y''(x,y). """
    y = empty(N)   # pylab's 
    (y[0], y[1]) = (y0, y1)
    dx2 = dx*dx
    for i in xrange(1, N - 1):
        x = x0 + i * dx
        y[i + 1] = 2 * y[i] - y[i - 1] + dx2 * d2y_dx2(x, y[i])
    return y

def shoot_QMharmonic_odd(Cn, ds=0.01, N=700):
    """ For a given "energy" Cn, return the wavefunction yn(s) for s>0 assuming an odd wavefunction """
    # This is one part of the "quantum shooting" algorithm.
    # Start at s=0 wity y(s=0)=0.0, y(s=ds)=ds, i.e. with slope 1,
    # and go far enough that the wavefunction blows up.
    return diffeq2(lambda s,y: (s*s - Cn)*y, x0=0.0, dx=ds, y0=0.0, y1=ds, N=N)

def shoot_QMharmonic_even(Cn, ds=0.01, N=700):
    """ Return yn(s) for s>0 assuming an even wavefunction """
    # similar to above, but start with slope=0 at s=0.
    # Doing this right means offsetting the grid by ds/2,
    # so using s=(ds/2, 2*ds/2, ...) so that the dy/ds=0 happens at s=-ds/2 and s=+ds/2
    # In other words, set x0=-ds/2, y0=1, y1=1.
    ### UNTESTED
    return diffeq2(lambda s,y: (s*s - Cn)*y, x0=-ds/2, dx=ds, y0=1.0, y1=1.0, N=N)

def count_crossings(yn):
    """ Given odd or even yn(s>0), return number of times it crosses y=0 for -inf < s < inf. """
    # Assume yn[s] is on a grid s=[0.0, ds, 2*ds, ...] and is only odd if 0th value is zero.
    crossings = 0
    last_sign = sign(yn[0])
    if last_sign == 0.0:
        parity = 'odd'
        last_sign = sign(yn[1])
    else:
        parity = 'even'
    for y in yn[1:]:
        s = sign(y)
        if s != 0.0:
            if s != last_sign:
                crossings += 1
        last_sign = s
    if parity == 'odd':
        return 2 * crossings + 1
    else:
        return 2 * crossings

def even(n):
    return n % 2 == 0
    
def QMharmonic(n):
    """ Return (Cn, yn[]), the n'th energy and wavefunction of the quantum harmonic oscillator """
    # Search via bisection for the output from shoot_QMharmonic_* which converges to 0 as s gets large.
    # The n'th state should cross the y axes n times, so we're looking for the transition from 
    # n to n+1 crossings.
    if even(n):
        shooter = shoot_QMharmonic_even
    else:
        shooter = shoot_QMharmonic_odd
    # 
    # These values work for small values of n, i.e. n=3.
    # To handle larger ones, increase Cn and N .
    low = {'C': 1.0, 'crossings' : count_crossings(shooter(1.0))}   # 1 crossing
    hi = {'C': 10.0, 'crossings' : count_crossings(shooter(10.0))}   # 5 crossings
    desired_C_accuracy = 1.0e-6
    loop_max = 50
    # print " debug: low[crossings] = ", low['crossings']
    # print "debug: hi[crossings] = ", hi['crossings']
    if low['crossings'] > n or hi['crossings'] < n:
        print " OOPS - you need to adjust the settings in QMharmonic to handle that n"
        return (-1.0, [0])
    loop_count = 0
    while True:
        loop_count += 1
        if loop_count > loop_max:
                print " OOPS - loop max hit in QMharmonic "
                return (-1.0, [0])
        C_guess = (low['C'] + hi['C'])/2.0
        crossings = count_crossings(shooter(C_guess))
        if crossings <= n:
            low = {'C': C_guess, 'crossings': crossings}
        else:
            hi = {'C': C_guess, 'crossings': crossings}
        # print " debug: low = ", low
        # print "debug: hi = ", hi
        # print
        if (hi['C'] - low['C']) < desired_C_accuracy:
            C = (hi['C'] + low['C'])/2
            y = shooter(C)
            return(C, y)

def DFT(y):
    """ Return Y[], the Discrete Fourier Transform of y[] """
    # A brute force calculation, just using the definition, i.e. N*N complex multiplications
    N = len(y)
    omega = exp( - (0+1j) * 2*pi/N )
    Y = zeros(N, dtype=numpy.complex)
    for k in xrange(N):
        for n in xrange(N):
            Y[k] += y[n] * omega**(k*n)
    return Y

def invDFT(Y):
    """ Return y[], the inverse DFT of Y[] """
    N = len(Y)
    omega = exp( (0+1j) * 2*pi/N )
    y = zeros(N, dtype=numpy.complex)
    for n in xrange(N):
        for k in xrange(N):
            y[n] += Y[k] * omega**(k*n)
    normalization = 1.0/N
    for n in xrange(N):
        y[n] *= normalization
    return y

def FFT(y):
    """ Fast Fourier Transform """
    # TODO
    pass

def infFFT(y):
    """ inverse Fast Fourier Transform """
    # TODO
    pass
In [2]:
# settings

# -- plot screen size --
# This is nominally in inches, but 
#  (a) rcParams['figure.dpi']=80 seems to be ignored, and
#  (b) the browser itself has its own zoom parameter anyway.
# The values here give 600x760pix (11x8.5 aspect) 
# for an apple cinema display (dpi=101)
rcParams['figure.figsize'] = (600/55.8, 460/55.8)

Running it

diffeq2 with the harmonic oscillator

In [3]:
# Test diffeq2 with simple harmonic motion : y'' = - y

dx = 0.1
x = arange(0.0, 2*pi, dx)
N = len(x)
y_harmonic = diffeq2(d2y_dx2 = lambda x,y: -y, N=N, dx=dx, x0=0.0, y0=0.0, y1=dx)

# For info on plotting, see e.g. http://matplotlib.org/examples/pylab_examples/simple_plot.html
plot(x, y_harmonic, 'ro', label='numerical')         # 'r' => red, 'o' => points as circles
plot(x, sin(x), linewidth=1.5, color='blue', label='exact')
xlabel("x")
ylabel("d^2(y)/dx^2 = - y")
title('simple harmonic motion')
legend()
Out[3]:
<matplotlib.legend.Legend at 0x10e284290>

The fit looks good by eye, even with a fairly large step. (That may well be because this diffeq Leaving the last one out, here's how the exact and approximate solutions differ.

In [4]:
hist( abs(y_harmonic - sin(x)) )
Out[4]:
(array([ 6,  8,  4,  7,  9, 11,  4,  7,  5,  2]),
 array([ 0.        ,  0.00024767,  0.00049535,  0.00074302,  0.0009907 ,
        0.00123837,  0.00148605,  0.00173372,  0.0019814 ,  0.00222907,
        0.00247675]),
 <a list of 10 Patch objects>)

If we didn't have an exact solution, the typical approach would be to look at how consistent the result is as dx gets smaller. Setting dx = dx/2 let's us compare every second point of the smaller grid with all the points of the bigger grid.

In [5]:
""" First, test pylab's statistics functions mean() and std() (standard deviation) """
a = arange(10)
sqrt(mean(a**2) - mean(a)**2) , std(a)
Out[5]:
(2.8722813232690143, 2.8722813232690143)
In [6]:
""" Looking at the accuracy of the harmonic oscillator numeric solutions """

def harmonic(dx, N):
    """ return a numeric solution to the harmonic oscillator with given dx """
    return diffeq2(d2y_dx2 = lambda x,y: -y, N=N, dx=dx, x0=0.0, y0=0.0, y1=dx)

def rms_difference_half_dx(dx = 0.1):
    """ Return standard deviation std() of the numerical approx run at dx and dx/2 """
    N = int(2*pi/dx)
    return std( harmonic(dx, N) - harmonic(dx/2, N*2)[::2] )

dxs = [0.001, 0.003, 0.01, 0.03, 0.1, 0.3]
errors = [rms_difference_half_dx(d) for d in dxs]
yscale('log')
xscale('log')
xlabel("dx")
title("harmonic : stand.dev. of ((dx solution) - (dx/2 solution))")
plot(dxs, errors, marker='o', linestyle='points')
Out[6]:
[<matplotlib.lines.Line2D at 0x10e2fa450>]

That's a straighter line than I expected. Looks like the accuracy goes like \(dx^2\).

diffeq2 with the exact pendulum

In [7]:
# Test diffeq2 with exact pendulum : y'' = - sin(y)

dx = 0.1
y0 = y1 =  3.0                # a bit less than straight up at y=pi
x = arange(0.0, 20.0, dx)     # an arbitrary xmax endpoint of 20, after one cycle by trial'n'error
N = len(x)
y_pendulum = diffeq2(d2y_dx2 = lambda x,y: -sin(y), N=N, dx=dx, x0=0.0, y0=y0, y1=y1)

# For info on plotting, see e.g. http://matplotlib.org/examples/pylab_examples/simple_plot.html
plot(x, y_pendulum, 'ro', label='exact pendulum')
xlabel("x")
ylabel("d^2(y)/dx^2 = - sin(y)")
title("exact pendulum")
legend()
Out[7]:
<matplotlib.legend.Legend at 0x10e483250>
In [8]:
""" Looking at the stability (accuracy?) of the exact pendulumsolutions """

def pendulum(dx, N):
    """ return a numeric solution to the harmonic oscillator with given dx """
    return diffeq2(d2y_dx2 = lambda x,y: -sin(y), N=N, dx=dx, x0=0.0, y0=3.0, y1=3.0)

def rms_diff_pendulum(dx = 0.1):
    """ Return standard deviation std() of the numerical approx run at dx and dx/2 """
    n = int(20.0/dx)
    return std( pendulum(dx, N) - pendulum(dx/2, N*2)[::2] )

dxs = [0.0003, 0.001, 0.003, 0.01, 0.03, 0.1, 0.3]
errors = [rms_diff_pendulum(d) for d in dxs]
yscale('log')
xscale('log')
xlabel("dx")
title("pendulum : stand.dev. of ((dx solution) - (dx/2 solution))")
plot(dxs, errors, marker='o')
Out[8]:
[<matplotlib.lines.Line2D at 0x10e4b61d0>]

Not as accurate as before, but looks pretty good for the smaller \(dx\). (This runs pretty quickly, but smaller \(dx\) values start to take longer.)

quantum shooting

First, lets test the method with a problem where we know the answer.

The quantum harmonic oscillator has evenly spaced energy levels which with the dimensionless units here are at \(C_n = (1, 3, 5, 7, ...)\).

Here's what a wavefunction looks like when we guess \(C_n = 4.0\) . \(y(s)\) starts at \(y=0\), arcs and crosses the axis before \(s=2\), and then heads down towards \(-\infty\).

In [9]:
ds = 0.01
s = arange(0.0, 6.0, ds)
N = len(s)
Cn = 4.0
y_shoot4 = shoot_QMharmonic_odd(Cn=Cn, ds=ds, N=N)
#
# plot guesses for wavefunction yn(s)
#
grid(b=True, color='grey')
xlabel("s")
ylabel("yn")
title("looking for state n=3 via quantum shooting")
axis(ymax=2.0, ymin=-1.0, xmin=0, xmax=6.0)
plot(s, y_shoot4, color="blue", label="wavefunction yn with Cn=4 : too low")
plot(s, shoot_QMharmonic_odd(Cn=8, ds=ds, N=N), color="purple", label="wavefunction yn with Cn=8 : too high")
plot([s[0], s[-1]], [0.0, 0.0], linestyle="dashed", label="y=0 axis", color="black")
legend()
#
## superimpose V(x) on the same graph - can see where curvature of y() changes at V=Cn
#
# twinx()
# ylabel("V(s)")
# plot(s, s**2, color="brown", label="potential V(s)=s**2")
# plot([s[0], s[-1]], [Cn, Cn], color="blue", label="energy Cn=4", linestyle="dashed")
# plot([s[0], s[-1]], [8, 8], color="purple", label="energy Cn=8", linestyle="dashed")
# legend(loc='upper left')
Out[9]:
<matplotlib.legend.Legend at 0x10e50d750>

Here's what we get solving for the 3rd excited state via shooting. The routine keeps going until of C is consistent to 1e-6. The calculated value isn't correct to that tolerance, but it's pretty close. The wavefunction looks good until near the end, when it starts to blow up. (These solutions always do that eventually, since they're of the form \(A exp(k x) + B exp(- k x)\), and even tiny roundoff errors in \(A\) eventually dominate.)

So ... it mostly works.

In [40]:
(C3, y3) = QMharmonic(3)
#
title("approximate 3rd excited state")
ylabel("y3[n] wavefunction")
xlabel("n")
plot(y3, label="y3[n] at C3={}".format(C3))
plot((0, len(y3)), (0, 0), color='black')
legend()
Out[40]:
<matplotlib.legend.Legend at 0x110883490>

OK, let's splice two of these together into an approximation of the real function.

In [42]:
y[500]
Out[42]:
0.0033362622474543942
In [43]:
y[550]
Out[43]:
0.032681361880761921
In [61]:
y_right = y[:500]
y_left = -y_right[::-1]  # reversed
y = concatenate((y_left[:-1], y_right))
plot(y)
Out[61]:
[<matplotlib.lines.Line2D at 0x110d2d450>]

discrete fourier transform

And so let's try our DFT on this thing.

First thing to notice: it's slow.

Second thing: all the power is in the lowest few components.

In [74]:
%time Y = DFT(y)
CPU times: user 8.14 s, sys: 10.7 ms, total: 8.15 s
Wall time: 8.15 s

In [76]:
plot(imag(Y), color="blue")
plot(real(Y), color="red")
Out[76]:
[<matplotlib.lines.Line2D at 0x111801b50>]
In [67]:
plot(real(Y[:30]), color="blue")
plot(imag(Y[:30]), color="red")
Out[67]:
[<matplotlib.lines.Line2D at 0x10efc4f50>]
In [68]:
plot(real(Y[-30:]), color="blue")
plot(imag(Y[-30:]), color="red")
Out[68]:
[<matplotlib.lines.Line2D at 0x110a28190>]
In [71]:
yy = invDFT(Y)
In [72]:
plot(real(yy))
Out[72]:
[<matplotlib.lines.Line2D at 0x11183ab10>]
In [73]:
plot(imag(yy))
Out[73]:
[<matplotlib.lines.Line2D at 0x11184ced0>]

Quick quiz:

* Which wavelengths are the non-zero parts?
* Why are only the imaginary parts nonzero,
  and why this form?
* Figure out by hand the wavelengths corresponding
  to the values of k which are nonzero.
* Using only the handful of nonzero Y[k] 
  for k near 0 or N, reconstruct an approximation
  of the original function.

Are we having fun yet?