SETI Algorithms project update

A quick update to list my progress with the SETI internship so far:

So far, most of the existing simple DSP code has been open-sourced in the form of a new library. I’ve named the library SETIkit, and you can find it on GitHub. The progress is documented in a wiki page.

The project uses CMake as the build system, and depends on

  • FFTW (Fast Fourier Transform in the West)
  • GSL (GNU Scientific Library)
The library is split into several modules
  • sq_utils contains several useful routines for logging, error handling, noise generation, and interop with the ATA (Allen Telescope Array) data format.
  • sq_signals has functions that generate test signals. For now, we have a routine named sq_gen_sine that streams out a complex sinusoid with gaussian noise.
  • sq_dsp contains a growing list of useful DSP utils:
    • FFT
    • Power calculation
    • Hann windowing
    • Conjugate calculation
    • DC offsets, Scaling, complex rotation
    • WOLA (Weighted OverLap Add)
    • Real/Imaginary channel extraction
    • Heterodyne
  • sq_imaging is a collection of a few functions that read signal data, scale it linearly/according to the power, and write it as a PGM image.
We now have all the code that is required to generate waterfall plots (more commonly known as spectrograms) of the SETI observation data.
Now there is a bunch of signal processing blocks that we use in a UNIX pipeline – programs that take signal data from one stream, do some processing, and pipe the resulting data to other programs.
A typical pipeline for generating waterfalls from the SETI data:
 cat ~/Data/2010-04-02-amc7-3693.4464-8bit-one-second.dat | ./sqsample -l 4096 | ./sqmix -c 0.0 
| ./sqwindow -l 4096 | ./sqfft -l 4096 | ./sqpower -l 4096 | ./sqreal -l 4096 
| perl -e 'while(read(STDIN,$bfr,4096*4)==4096*4) {syswrite(STDOUT,$bfr,1024*4,1536*4)}' 
| ./sqpnm -c 1024 -r 1200 -p > out.pgm
This takes data from a file that contains one second’s worth of observations of the AMC-7 geosynchronous satellite’s broadcasts.
  • The sqsample block takes 2-channel (quadrature) 8-bit data from stdin, and writes out samples (floats) to stdout in the form of alternating real and imaginary values.
  • The sqmix block heterodynes a chosen digital frequency (here 0.0 Hz, we’ll see more later)  to DC (0 Hz). That is, you can use this to move the spectrum to the left and right.
  • The sqwindow block applies a hanning (or hann) window to the signal.
  • The sqfft block streams out the Fast Fourier Transform of the windowed signal.
  • sqpower computes the instantaneous power signal.
  • sqreal extracts the real part from the information.

The 4096 argument to all those blocks is the number of samples they can read, write, and process in one go.

The perl code neatly arranges the signal data so that it can be visualized as an image, also giving you control over the ‘region’ of the image you want to see. The sqpnm block then takes this data, scales it, and writes it out to a binary PNM/PGM image.
Here’s the waterfall that the above pipeline generates:

Waterfall Plot. Horizontal: Frequency | Vertical: Time | Brightness: Frequency Intensity.
That’s it for now.