plot a huge amount of data points

Question

I have encountered a strange problem: when I store a huge amount of data points from a nonlinear equation to 3 arrays (x, y ,and z) and then tried to plot them in a 2D graph (theta-phi plot, hence its 2D).

I tried to eliminate points needed to be plotted by sampling points from every 20 data points, since the z-data is approximately periodic. I picked those points with z value just above zero to make sure I picked one point for every period.

The problem arises when I tried to do the above. I got only a very limited number of points on the graph, approximately 152 points, regardless of how I changed my initial number of data points (as long as it surpassed a certain number of course).

I suspect that it might be some command I use wrongly or the capacity of array is smaller then I expected (seems unlikely), could anyone help me find out where is the problem?

def drawstaticplot(m,n, d_n, n_o):
    counter=0
    for i in range(0,m):
        n=vector.rungekutta1(n, d_n)
        d_n=vector.rungekutta2(n, d_n, i)
        x1 = n[0]    
        y1 = n[1]
        z1 = n[2]
        if i%20==0:
            xarray.append(x1)
            yarray.append(y1)
            zarray.append(z1)
    for j in range(0,(m/20)-20):
        if (((zarray[j]-n_o)>0) and ((zarray[j+1]-n_o)<0)):
           counter= counter +1
           print zarray[j]-n_o,counter
           plotthetaphi(xarray[j],yarray[j],zarray[j])

def plotthetaphi(x,y,z):
    phi= math.acos(z/math.sqrt(x**2+y**2+z**2))
    theta = math.acos(x/math.sqrt(x**2 + y**2))
    plot(theta, phi,'.',color='red')

Besides, I tried to apply the code in the following SO question to my code, I want a very similar result except that my data points are not randomly generated.

Answer 1

Shiuan,

I am still investigating your problem, how ever a few notes:

Instead of looping and appending to an array you could do:

select every nth element:

# inside IPython console:
[2]: a=np.arange(0,10)

In [3]: a[::2] # here we select every 2nd element.
Out[3]: array([0, 2, 4, 6, 8])

so instead of calcultating runga-kutta on all elements of m:

new_m = m[::20] # select every element of m.

now call your function like this:

def drawstaticplot(new_m,n, d_n, n_o):
    n=vector.rungekutta1(n, d_n)
    d_n=vector.rungekutta2(n, d_n, i)
    x1 = n[0]    
    y1 = n[1]
    z1 = n[2]
    xarray.append(x1)
    yarray.append(y1)
    zarray.append(z1)
    ...

about appending, and iterating over large data sets:

append in general is slow, because it copies the whole array and then stacks the new element. Instead, you already know the size of n, so you could do:

def drawstaticplot(new_m,n, d_n, n_o):
    # create the storage based on n,
    # notice i assumed that rungekutta, returns n the size of new_m, 
    # but you can change it.
    x,y,z = np.zeros(n.shape[0]),np.zeros(n.shape[0]), np.zeros(n.shape[0])

for idx, itme in enumerate(new_m): # notice the function enumerate, make it your friend!
    n=vector.rungekutta1(n, d_n)
    d_n=vector.rungekutta2(n, d_n, ite,)
    x1 = n[0]    
    y1 = n[1]
    z1 = n[2]
    #if i%20==0: # we don't need to check for the 20th element, m is already filtered...
    xarray[idx] = n[0]
    yarray[idx] = n[1]
    zarray[idx] = n[2]
    # is the second loop necessary?
    if (((zarray[idx]-n_o)>0) and ((zarray[j+1]-n_o)<0)): 
       print zarray[idx]-n_o,counter
       plotthetaphi(xarray[idx],yarray[idx],zarray[idx])