Tawhiri/start

Tāwhirimātea

Tawhiri is the name given to the next version of the Landing Prediction Software, which will probably be different enough from the current version (see below) to warrant a new name.

The name comes from a Māori god of weather, which rather aptly "drove Tangaroa and his progeny into the sea" (WP).

Introduction

Long story short, the predictor consists of

• Some python responsible for downloading NOAA wind data.
• Some C responsible for solving a simple DE; Forwards Euler and assumes the balloon moves at the speed of the wind (it is still very very accurate, so not much effort is required there).
• Some PHP responsible for drawing a map.

And the motivation for this project is

• The servers the python relied on were getting really slow, so that could be rewritten from scratch (done).
• The C is old, and we want to add new features like:

    * Monte Carlo prediction, better integration
    * a floating balloon mode
    * perhaps something that takes into account the altitude of the ground when it lands
* It would be easier to integrate the various bits if they were all written in Python. This would also ease integration with other projects too, like [[http://spacenear.us|the tracker]] (slated for a rewrite in Python) and [[http://habitat.habhub.org|habitat]] (Python; CUSF).
* PHP is disgusting.

Notes

• The project page for the previous version of the predictor has an extensive general overview
• Some notes on how the code Current Predictor (github/cusf-stanadlone-predictor/pred_src) works.

Ideas

• Rewrite the C portion of the predictor in Python
• New / better integration
• Different altitude model options
• Replace PyDAP

Summer 2013

In the Summer of 2013, PyDAP broke. The servers became too slow to use properly.

Daniel (djr61) hacked up in 2 days a replacement wind downloading program in python (gevent, numpy), that gets binary (GRIB) data off the NOAA FTP servers instead, and unpacks it to a 18GB array of doubles. The C then accesses that by memmapping the entire file into its address space and casting it to an array of the right dimensions.

This works hilariously well with predictions now completing in under 20ms. We have gigabit internet, lots of spare power and loads of RAM, so it's not as bad as it sounds. It also hugely simplified the C code.

See my post to the UKHAS mailling list for a quick summary (hourlies)

This is deployed at http://predict.habhub.org/.

Some Notes:

It was a 2 day sprint. There are no comments. No unit tests. It works, and the interaction of the downloader with the other bits should be fairly minimal. I intend to eventually clean it up.

Code:

• The clean, new, predictor tawhiri
• The C, PHP + various deployment bits: tawhiri-oldui-juryrig. This repository is a fork of cusf-standalone-predictor that jury-rigs the new downloader into the old code.

Tasks

1. Rewrite the core predictor (in Python?)
2. Add a float mode
3. Add better integration
4. Rewrite the web interface (Python? gevent? Flask? PostgreSQL? Worker daemons?)

People

• Daniel (djr61)
• George (gd365)
• Ilya (im354)
• David (db590)
• Thomas (tp378)