It's all in the application: Recon Instruments Snow 2 vs Paragliding: Part 9

Continued from Part 8

Now that we have a result, it is time to display it to the user.

The Displays are implemented as disjoint Fragments. Each Fragment has no knowledge of any other - nor does the 'master' Activity have any knowledge (at a code level) of the Display Fragments it contains.

Each display has a Broadcast Receiver that listens for Flight Data messages from the Flight Data Service. When a message arrives, an AsyncTask is created to invoke each of the Displays Processors. The result of each Processor is cached and ultimately the Display is updated.

The single deviation from this system involves the Compass. The Compass updates in real time as the user turns their head. Moving the Heading position by way of a Flight Data Message via the Flight Data Service would involve too great a latency at this point (this could be over come with predictive interpolation - but would add complexity with little benefit*). So, the Heading data used by the Compass is fed directly into the Compass Display by way of callback. Processing in the callback is throttled to a period no faster than 30 ms. When an update occurs, each of the children of the Compass (CompassSubDisplays) are walked, their Processors invoked, and the results displayed.

* that said, thought has been given to implementing a complete Flight Data Recorder that would include Head Orientation. The would allow for future playback and enable the creation of PiP video with an overlay of what the pilot would have seen on the HUD.

It's all in the application: Recon Instruments Snow 2 vs Paragliding: Part 8

Continued from Part 7

The refactor continues.

With raw Channel* data being broadcast by the Flight Data Service, we still need to process it before it can be displayed to the user -> either due to the 'noisy' nature of the data (such as a fluctuations in instantaneous vario readings) or the need to aggregate multiple Channels (such as determining Wind Drift or Thermal Location).

The Processors exist to do these tasks of aggregation and low pass filtering. As Channel data arrives at the Display, that data is fed to each of the Processors that the Display has instantiated. If the Processor has all of the data it needs, a result will be returned. If insufficient time has passed since the last invocation of the Processor, the Processor will simply return the cached last result.

Processor

Processors come in two forms:

1. Framed.
2. Custom. 

Framed Processors are built from an aggregate of operations:

1. Converters.
2. Adjusters.

A Converter takes two or more raw Channels and performs an action upon them - such as subtraction or division - to then return a single result.

Adjusters, on the other hand, take a single raw Channel and perform an action upon it. Think low pass filtering as one example.

Converters and Adjusters

Framed Processors typically involve simple operations that do not involve complex computation or access to external data sources beyond the raw Channels.

For more complex computation, one would implement a Custom Processor. Examples would include Wind Drift and Waypoint Navigation.

Custom Processors

All of the Processors are obtained via the Processor Factory. This factory abstracts the need to differentiate between the Framed and Custom implementations. Requesting a Processor from the Factory is done via the Processor ID.

Processor IDs

With the Processors up and running, we can now Display the results.

* A Channel is a single raw stream of data coming directly from the device - such as vario, pressure or gps altitude, latitude, longitude, ground speed.

Onto Part 9: Displays.

It's all in the application: Recon Instruments Snow 2 vs Paragliding: Part 7

Continued from Part 6

A refactoring we will go.

In an attempt to better segregate the data provision from the data consumption, I've moved the process of flight data collection and broadcast into a service running under its own thread.

Providers

The initial source for the raw flight data (referred to as Channels) are the Providers which are invoked by way of either system callback or periodic runnable.

There are three groups of providers:

1. Android Generic - Internal GPS and Battery.
2. Recon Specific - Sensors and Head Orientation.
3. Test.

The Providers all derive from the abstract Provider class and are expected to implement start/stop, the (de)registration of client, and configuration information (identification and channels supported).

Providers

*Note: At the time of this writing, the Head Orientation Listener does not go through the following flow - it is instead part of the composition of the Compass Display (due to the need for minimal latency).

Multiplexer

As Channel data is received by each provider, it is fed to the registered client of the provider. In this case, each Provider shares the same client - the Message Multiplexer. When more than one Provider is a source for a single Channel, the Multiplexer determines from which Provider the Channel will be ultimately sourced.

Providers are ranked based on registration order - earlier Providers getting priority over later registrants.

When a Channel arrives at the Multiplexer, we first check to see if the Channel has already been reserved by a Provider. If so, is the current Channel Provider of higher priority? If yes, now adopt this Provider of the Source and pass on the Chanel Data. If no, do nothing. If the Channel is not reserved, then adopt and pass on the Channel to the Client.

Providers also have the ability to deregister their reservation of a Channel in the event they go offline.  If the Provider has a Channel reserved in the Multiplexer and sends a channel failed message, the reservation is cleared and we treat the channel as unreserved.

Cache

As Channels pass through the Multiplexer, they are stored in the cache until such time as the Flight Data Service can bundle and broadcast them.

Multiplexer and Message Cache

Flight Data Service

Containing this whole system is the Flight Data Service. It is a simple extension of the standard Android Service with the addition of its own thread of execution. The Service periodically (every 0.5 seconds) bundles Channel data stored in the Message Cache into an intent and broadcasts it.

Flight Data Service

The Data Service contains the Providers, which the Service registers with the Multiplexer. The Multiplexer is then in turn registered with the Message Cache. The Providers are then started and Channel data is bundled and broadcast for the Displays to consume.

Refactor Continued ...

It's all in the application: Recon Instruments Snow 2 vs Paragliding: Part 6

Progressing along - next up, thermal core location - using GPS positioning coupled with variometer readings to determine the most likely position of the thermal core in relation to the pilot.

The most correct version would probably entail a least fit squares/orthogonal linear regression ala Eberly. Two challenges emerge if one goes this route:

1. Implementation Complexity - the operations involved (including a descending Eigen vector sort) are complex and would be a nightmare to debug.
2. Execution Complexity - the operations are math intensive. Either reduce the frequency of execution (and the accuracy of the result) or shuttle off to the GPU via shader computation.

The decision -> implement first, optimize second. Find a simpler implementation to begin with, evaluate, then decide if a more robust solution is needed.

That said, it doesn't mean one needs to completely ignore how the OLR solution goes about accomplishing its goal. In particular, the use of a centroid that acts as the point through which the best fit vector passes. 

The centroid is just the average of the points. Unmodified it would simply demarcate the middle of the sample set. But it we were to weight it with, oh, the vario data we could bias the position towards the side of the sample set with the strongest climb values. 

Weighting is a matter of determining the strongest vario value in the set:

for (ThermalSample sample : mSamples.values()) {
    if(sample.mVario > mVario)
        mVario = sample.mVario;
}

... and then reducing the weight of all other points by a geometric factor based on the difference. Samples with a vario reading equal to or less than zero are ignored and not added to the sample set.

Double weight = 1.0 - (mVario - sample.mVario) / mVario;
weight *= weight;

The resulting centroid calculation:

for(ThermalSample sample: mSamples.values()) {
    Double weight = 1.0 - (mVario - sample.mVario) / mVario;
    weight *= weight;
    weightSum += weight;
    altitudeSum += weight * sample.mLocation.getAltitude();
    latitudeSum += weight * sample.mLocation.getLatitude();
    longitudeSum += weight * sample.mLocation.getLongitude();
}

altitudeSum /= weightSum;
latitudeSum /= weightSum;
longitudeSum /= weightSum;

This calculation is done frequently as the sample dataset continually updates. To prevent data from going stale, pruning is done as new elements are added - currently based on age (> 60 s) and distance (> 200 m):

for(Iterator<Map.Entry<Double, ThermalSample>> it = mSamples.entrySet().iterator(); it.hasNext(); ) {
    Map.Entry<Double, ThermalSample> entry = it.next();
    Location location = entry.getValue().mLocation;
    if(location.distanceTo(currentLoc) > MAX_SAMPLE_DISTANCE || time - location.getTime() > MAX_SAMPLE_AGE) {
        it.remove();
    }

A far less complex solution vs. OLR and appears to give reasonable results with the test data. Displaying the result is done where the MFD used to sit. This new 'map' display is rendered using simple Android Canvas operations and can be purposed beyond simply displaying the current core guesstimate. The glide ratio and vario reading now have a permanent display just above the old MFD location - though smaller in size.


Thermal core is just ahead and to the right. 
Green means core strength is less than 2x current climb rate.
Battery remaining in lower right corner.
Core further away. 
Red means core strength is more than 4x current climb rate. 
Yellow is used for 2 to 4 x.
Vario is just above (currently reading 0.0 m/s).

The Refactor begins...

It's all in the application: Recon Instruments Snow 2 vs Paragliding (Part 5)

With the Christmas break spent optimizing my BLE implementation (to get some degree of reliability in messaging from the Bluetooth enabled vario), I figured I was set to begin field testing with my upcoming trip to Brazil.

Then an email from the trip organizer -> we will be using turn points to keep everyone close to roads, making for easy retrieve. 

Looks like navigation support just got bumped in priority.

The turn points were distributed via the standard .wpt file format. I just needed a way for one to select the points, assign radii, display bearing and distance to the next point, and pop them from the queue as they are reached.

A split implementation seems best -> build the list on the phone and push to the HUD.

The HUD side: 

The red inverted triangle shows the bearing to waypoint that is 4.4 km away. 

The MFD has been updated to include the height above launch to its cycle of information when within 5 km of launch.

A short while later while on bar and gliding away from the turn point (looking back at it), a wind guesstimate shows up. Heuristics need tuning.

Flight time is in the lower left corner.

The phone side:

Turn points from .wpt loaded and parsed.

Launch point selected, now able to add to waypoint list at bottom

'Add' selected, enter the radius in meters.

Launch and landing added - aggregate non-optimized distance displayed. 'Save' is now available. Selecting a waypoint in the bottom list will replace the 'Add' button with a 'Remove'.

The Save has been re-purposed to 'View' for now. Launch visible with the turn point radius in green.

...and the LZ. The black line is the path to be taken. Again non optimized.

Once saved, the waypoints (along with associated turn points and radii) are pushed to the HUD via Wifi P2P. The HUD has a thread watching for the turn point file to be updated and will pick it up when the upload is complete. The HUD will then populate the navigation queue and display the next waypoint on the compass (as above).

Next steps include the ability to slide the vertices of the flight path around within the turn point cylinder radii (hand optimization). Following that would be automated optimization using some form of the Travelling Salesman genetic algorithm.

Continued in Part 6

It's all in the application: Recon Instruments Snow 2 vs Paragliding (Part 4)

BLE - Bluetooth Low Energy.

Its all the rage in 'wiring' up sensors to a flight data aggregator (such as the GliderHUD on the Recon Instruments Snow2).

Now that the GliderHUD app can display flight data generated by the internal GPS, its time to look at external data sources like the XCTracer vario/GPS.

Using the sample application provided by Recon (which is based on the Android BLE tutorial published by Google) - I was able to get the Snow2 and XCTracer chatting with one oddity. It appears that the BLE stack on the Snow2 struggles with multipart messages. This is not particularly surprising as the only BLE device the Snow2 could talk to originally was the remote. I imagine that the up/down state of 6 buttons could easily be packed in the 20 byte limit of a BLE message - precluding the need to address a rapid chain of messages hitting the onboard transceiver and protocol stack. Just a hunch though. It could also be the quality of the Bluetooth chip itself. Who knows.

Well, the message format I wanted to support out of the box was the native XCTracer frame with a plethora of data.

But it is not to be.

After the third (sometimes fourth) XCTracer subpart, a new message begins - meaning the remainder of the original is dropped. Trying to accommodate a rapid turn around within the characteristic notification callback, I tried posting the byte data to a Handler immediately. Still no dice. Maybe a bug on my part.

What I do know works (mostly...mostly) is the LXWP0 format - it comes across in two parts as opposed to the five or six portions with the XCTracer sentence.

So LXWP0 it is.

Sadly this costs me the groundspeed - it would be nice to be able to skip using the onboard GPS on the Snow2 altogether except as a backup.

The changes are up on GitHub - https://github.com/Levemus/GliderHUD

The vario data coming from the XCTracer via BLE (the XCTracer is reporting the altitude as a negative, which the altitude display floors at 0 m):

Continued in Part 5

It's all in the application: Recon Instruments Snow 2 vs Paragliding (Part 3)

A quick recap from Part 1 and Part 2:

A killer deal on a Recon Instruments Snow2 +
A device that runs on the Android OS +
A paragliding dev with a modest amount of experience in game engine implementation on Android.
=
A Paragliding HUD work in progress.

Using the sample Compass app from Recon and some ideas on wind drift approximation from Alistair Dickie (along with a circular regression implementation from Dr. Micheal Doube) - I managed in the last pair of posts to put together a simple PG HUD that showed heading, bearing, altitude, groundspeed, and wind drift direction (along with a poor mans climb rate/vario and glide ratio).

In the 'rush' to slam together this prototype implementation, things became a bit of a hot mess (as they always do) - with the inevitable intermixing of UI and data provision (emmveecee rather than MVC, as it were).

Reorganizing

So a refactoring we will go (caveat: it is a work in progress, this is simply a snap shot at the current moment).

The result: The display elements are listeners that subscribe to broadcasters.

Broadcasters and Listeners

When the app is started, broadcasters and display elements are instantiated. The broadcasters are passed to each display (listener).

The listener attempts to register itself with the broadcaster, specifying which Flight Data elements it cares about and the minimum interval between notifications.

The broadcaster looks at the requested elements, compares them against what it can provide and replies with those it can service.

The listener stores the outstanding services it still needs until the next broadcaster is sent its way to register with.

Flight Data

Once registration is complete, the listeners wait for invocation of their onData callback. The passed param is a Flight Data object. The listener can query the Flight Data object for values it needs to update itself with. If the Flight Data object cannot provide the value (as could be the case when subscribing to multiple broadcasters providing different data) - an exception is tossed (the alternative could a TryGet like function seen in C# dictionaries, perhaps).

With data in hand, the listener (display) can update and refresh itself.

ListenerBroadcasters

Some broadcasters are aggregators of data - subscribing to get information, processing it, then providing a result to a downstream listener. The WindDrift is one such case - it subscribes to the GPS for bearing and ground speed information, processes it, and provides wind speed and direction.

Moving forward

Right now the implementation does not handle competing data broadcasters - we could get altitude information from the onboard Android GPS and we could (once implemented) get that same altitude from a BLE device such as the XCTracer. Experimentation leads me to believe that the XCTracer will provide better altitude data, but being a BLE device - it is not as reliable a source as the onboard GPS. This needs to be accounted for - some form of override and failsoft -> use the Android GPS for altitude until a better source appears, use that source until it drops off, then back to the Android GPS. Maybe an intermediary manager.

Source

Disclaimer:
Source is up on github, purely for educational purposes. Paragliding is a dangerous activity and use of this software in any way, shape, or form, can result in serious injury or death. Levemus Software Inc. assumes no responsbility and provides no guarentee related its use. Any use of the source must include the below github url from which the source originated. Any commercial use must have prior written permission from Levemus Software Inc.

https://github.com/Levemus/GliderHUD

Continued in Part 4

It's all in the application: Recon Instruments Snow 2 vs Paragliding (Part 2)

While it has been hectic week as we try to roll out a new toolset to the content creators, things are under control to the point that I get to have my weekend. To be honest, I'm pleasantly surprised at how little OT this return trip to the games industry juggernaut EA has involved (compared to my last stint a decade ago). *knock on virtual wood*

In any case, free time => back to the repurposing of a Recon Snow2 HUD for paragliding.

In the last post, I managed to get the device to show heading, altitude, ground speed, glide ratio, and climb rate. Now time for something a little more meaty - wind drift.

Wind drift is a rather useful morsel of info. It can be 'back of the hand' calculated by eyeing the ground speed while performing a complete turn - note the heading and degree with which ground speed drops off to its lowest value. That method works just fine if there are no obstacles (such as other pilots) nearby.

But I am lazy. Also when there are airborne obstacles nearby, my line of sight intersects the screen of my Flymaster with a frequency of about....oh....zero.

So having the digital navigator compute and display the value for perusal at my leisure is a big plus.

Before beginning: a big hat tip to Alistair Dickie @ BlueFlyVario and Dr. Micheal Doube @ the Royal Veterinary College. Without the prior works of these two gents, I'd be stuck doing the back of the hand method.

The baseline assumption: A paraglider has a very narrow range of speed compared to almost every other form of aircraft => the delta between trim and stall can be as narrow as 15 kph. This means we can treat airspeed as near constant.

An aircraft at a constant speed and at a constant bank will fly in a near perfect circle absent any outside influence (such as the wind). If one were to take the tangental ground speed velocities (constant in magnitude, varying in direction) and plop them down such that they share a common base - we could draw a circle connecting the tips of these ground velocity vectors. The radius of the circle would be the airspeed and in the case of zero wind it would match our ground speed.

In otherwords, if our PG has an airspeed of 30 kph and there is nil wind, our groundspeed will be 30 kph. Likewise, where we are facing is were we are going (no drift).

If we were to introduce wind, we know our ground speed will vary (facing into the wind will reduce our ground speed, running with the wind will increase it). But our airspeed will not change.

Back to our PG. 30 kph airspeed. 10 kph straight on headwind. Ground Speed? 20 kph. Likewise, 30 kph airspeed. 10 kph straight behind tailwind. 40 kph ground speed.

The simplest of vector addition.

What about crosswind? It is still vector math (but will not involve a little trig to determine direction).

Airspeed is still 30. Crosswind is perfectly square on to our side (90 degrees off our heading) and 10 kph.

What is our ground speed (and direction)?

Treat the heading as the Y component, treat the wind as the X component.
Y = 30 kph.
X = 10 kph.

The magnitude of a vector is the square root of the sum of the squares of its components.
Huh?

This: square root ( 30 * 30 + 10 * 10) => square root (900 + 100) => square root (1000) => 31.62 kph.

Our ground speed will be 31.62 kph. But not in the direction we are facing.

How do we find the direction?

We know from high school that the sine of an angle in a triangle is the ratio of opposite over hypotenuse. The cross wind (10 kph) is the opposite edge, the hypotenuse is our just computed ground speed (31.62 kph).

sin angle = 10/31.62 => 0.3162.
Inverse sin (arcsin) 0.3162 and we have our angle (in radians) => .31172129 rads.
In degrees: rads / PI * 180 => 18.4 degrees.

Because of the wind: we will travel @ 31.62 kph, 18.4 degrees off our heading.

A long winded way to say that the wind impacts not only the speed but also our direction of travel in the air.

That is all fine and dandy, but what if we don't know the wind component and only know the end result (only ground speed and a bearing). How do we determine the wind component?

Circular regression.

Huh?

Given a constant airspeed and bank angle (rate of turn), one can create a circle that best fits the tips of the ground speed vectors that share a common origin. The vectors will vary in magnitude due to the wind. This variation will mean the common point of original for the velocity vectors will not be the centre of the circle. This is very important, as it is this delta that we want.

A picture would help immensely here, but that will likely have to wait. Trust me.

One method of finding the best fit circle to a trio or more of non-colinear points is the Taubin-Newton Circular Regression algorithm. This is what we will use.

The regression gives us the centre and radius of the best fit circle. The radius will be our airspeed. The centre will be offset from the common point of origin. The magnitude of the vector joining the centre of our newly computed circle and the common origin of the wind drift influenced ground speed vectors will be the wind strength. The direction of this end vector will be line of wind travel.

We now have a wind direction and speed. Easy peasy in the words of a certain Brit SIV instructor.

In testing, I assume a 30 kph airspeed with a 10 kph straight south wind. The Y axis is North/South with North being positive. The X axis is East/West with East being positive.

The velocity vectors (X, Y) fed in look like:
(0, 40)
(0, -20)
(30, 10)
(-30, 10)

The output vector indicates wind out of the south @ 10 kph.

The result:

The blue arrow represents wind direction. I have yet to add a strength value above it. The bearing indicator (green) is being obscured as the line of travel is in direct opposition to the wind (there a pixel thick line of green just to the left of the blue wind direction indicator).

Next steps involve retaining the wind values over time, applying an exponential weighted average over these retained values to smooth out any sudden changes (and give more weight to more recent data), and optimizations.

Continued in Part 3.

It's all in the application: Recon Instruments Snow 2 vs Paragliding.

Over the summer an online dealer in Utah for the Recon Instruments family of devices had the Snow2 + Scott googles on sale for over 50% off. Even with the exchange rate, it was a phenomenal deal compared to the price local dealers were charging. The Snow2 is a pseudo HUD in that it does not directly overlay ones view, but instead sits on a small display in the lower right field of view.

An image of what the HUD looks like within the goggles, from Necessary Coolness.

Here is a link to Recons info page.

The HUD software is geared almost exclusively towards snow sports (thus the name Snow2 and likely why I got it so cheap mid summer). But it runs on Android, and strangely enough yours truly has a decent amount of experience with Android fairly close to the metal.

Trivial to repurpose this device towards a more paragliding centric application. Grab the Recon SDK, the sample app, and a hacking we will go.

The result:

Fairly simplistic. It autostarts on boot, precluding the need for the all-too-easy-to-lose remote. The compass rotates in line with heading, the green arrow indicates bearing. If the user has a sufficient change in bearing or a climb rate above 0 m/s, the centre display will show climb rate, otherwise it shows glide ratio. Climb and glide are averaged.

Next up will be determination of wind direction and speed (a blue arrow on the compass?). After that, maybe speed to fly, maybe BLE connectivity (if Recon has finally updated the OS for the device with proper SDK support) to a PGing GPS such as the XC Tracer.

In any case, it and the laptop will follow me to Brazil for further testing/dev.

Continued in Part 2.

...Or skip ahead to the current refactor.