HDZero VRX – An In-depth Technical Review

But first, a quick recap…

It’s hard to believe it has been 9 years since I last posted to this blog, but I feel writing this technical review of the HDZero system is exactly the motivation I need to give blogging another go. Firstly, to fill in the massive gap since my last post in Aug 2014 about porting the SimonK firmware to an Atmega-88 ESC, … basically my interest in First Person Video (FPV) drones took off and I’ve become engrossed in the hobby ever since. I even occasionally post freestyle videos on my KandredFPV YouTube channel. On top of that that we had another 2 kids (taking us to 3) and my free time quickly disappeared.

The digital evolution of FPV

I’ve always had an interest in RC but only got into FPV after struggling to pickup LOS (line of sight) flying when I built my first quadcopter. After being terribly disappointed at how difficult it was to fly the darn things in LOS/Acro, I would later stumble across some FPV video on YouTube of pilots flying with ease and having a great time. I was convinced it was the solution to all my problems, and rightly so because I fell in love with the hobby and quickly got to work at becoming a less sh*t pilot.

Until recently, all of my experiences of FPV had been using analog video which is just another way of saying a Composite Video encoded signal (CVBS) is transmitted from the quad to the pilot via a video transmitter & receiver using VSB RF modulation (a form of narrowband-AM, yes AM!) in the 5.8Ghz band. The CVBS signal can be either PAL (576i@25 fps) or NTSC (480i@30 fps), which are both considered standard definition video but with all the visual artifacts and defects which normally accompany a legacy analog broadcast TV system.

Enter the HDZero system – formerly known as SharkByte

About a year ago I finally decided digital had matured enough to give it a try and I bought the Race VTX and HDZero SharkByte VRX for use with my old Dominator HD2 goggles, and I was blown away by how much of an improvement this system was over analog.

How it works

Digital RF communications

At the heart of any digital FPV system is a digital RF transmitter and invariably the modulation used is some form of the OFMD modulation scheme because of its unique ability overcome the inter-symbol interference (ISI) and channel fading caused by multipath propagation. To be clear, ISI and multipath fading affects all RF communication systems. The effects of multipath fading are normally mitigating using pre-trained equalisers to reverse the channel distortions, and in digital RF systems, low symbol rates and guard intervals help with ISI. However, multipath fading is more problematic in mobile RF applications (i.e. where at least one endpoint is moving relative to the other) such as FPV; since the environment is constantly changing and the equalisers need to be constantly retrained. OFDM allows for this by integrating the pilot signals used for equaliser training into the modulation scheme as subcarriers, so the fading correction can be adaptive.

It’s worth noting that MIMO-OFDM is another variation of this technology (found in 4G+ and 802.11n+ devices) which actually leverages multipath propagation to transmit multiple distinct channels simultaneous via different paths (a.k.a spatial multiplexing) resulting in an increased channel capacity. As far as I’m aware, neither HDZero, DJI or WalkSnail uses this technology, since it’s normally used with linearly polarised diversity antennas and matching receiver and transmitter antenna counts.

The Divimath SoC

Parsing through the documentation on the Divimath (the creators of HDZero) website, I found the datasheet brief for their DM5680 chip which confirms their use of OFDM in the system.

With the exception of the system’s lack of any kind of video compression – which Divimath states there system does not use – it actually bares my similarities to Digital Video Broadcating – Terrestrial (DVB-T) system which replaced analog broadcast TV here in New Zealand (and most of the world) since back in 2008. And like DVB-T, HDZero appears to be a broadcast video system, since no point-to-point (i.e. binding) association is required between the VTX and VRX. You simply select a channel and you immediately start receiving the FPV broadcast, which conveniently allows for any number of spectators to tune in.

Divimath has documented the HDZero system as having a bandwidth of 27Mhz, and we know the RF channel bitrate to be 100Mbps because Carl, the founder of Divimath, mentioned it as part of a technical walkthrough of the system in Chris Roser’s YouTube interview. We don’t know the OFDM symbol rate (i.e. symbol duration and size of guard interval). We also don’t know what size of QAM is used (i.e. 16 or 64) with the OFDM data subcarriers or the number of subcarriers used; DVB-T uses 1700+ carriers, but my understanding is using such large IFFT bin size places a higher demand on the DSP, so it’s probably likely down in the 10s if not 100s of sub-carriers similar to WiFi (802.11x).

There is no evidence of the bitrate (and therefore video encoding) being adaptive based on SNR, although Carl suggested as much in that interview. This functionality would require SNR (which is collected at the receiver) to be sent as telemetry in back-channel to the VTX, and while the DM5680 chipset does appear to include a full transceiver, Mads Tech’s DJI vs HDZERO RF Deep Dive YouTube video saw no such telemetry being sent in a spectrum analysis of HDZero. In my opinion, HDZero would need to be a point-to-point system like DJI in order for this work since it would not be possible to adaptively vary bitrate for multiple VRX spectators in a broadcast system. So spectator mode would need to be separate multipoint RF links with their own independent bitrates, which we know it is not.

A screenshot of HDZero’s spectrum from Mad Tech’s YouTube video

VRX Hardware

Livyu FPV has done an excellent hardware teardown of the Shark Byte VRX which I highly recommend if you haven’t seen it already. I’ve captured most of those details in the diagram below, and filled in a gap he was uncertain of, i.e. the H.264 DVR encoder. At the heart of the VRX is a XILINX Spartan-6 FPGA which if I had to guess is running some kind of ARM soft-core for executing the firmware in addition to what appears to be Divimath’s own image fusion algorithm. This is because, while each omni/patch antenna pairs (on the left and right) are setup as diversity inputs to the AD9361 chips, they each feed independent DM5680 OFDM baseband video decoders, and thus generate two separate feeds of YUV420 image data into the Spartan-6. So Divimath must be fusing the frames of these images together somehow, either at a low-level by selecting uncorrupted 8×8 blocks or, at a higher level by selecting the image frame with the least corruption, then outputting this to the HDMI driver and optionally the H.264 DVR encoder.

This is good to know because there might be some advantage to using different orientations for the left and right omni antennas to promote image diversity, rather than mounting them both upright.

The Video Codec

What sets HDZero and DVB-T apart is Divimath’s proprietary low latency video encoding algorithm which is used in place of the higher latency traditional MPEG codec. Well to be precise, Divimath’s website says “Video Coding: No video enceder” which I believe we should interpret as “no traditional video encoder”, since it’s quite easy to see that there is no way the system can transmit 720p60 video in a 100Mbps channel.

Assuming 4:2:0 chroma subsampling, a common image compression format used in digital cameras whereby each pixel is encoded as 12 bits (instead of 32), for 720p60 video we can calculate the uncompressed bitrate as:

=> width x height x bits per pixel x frames per sec
=> 1280 x 720 x 12 x 60 = ~664Mbps

This is over 6 times the RF channel capacity, so some form of compression must be used.

Video Compression Theory

There are two aspects of video compression: spatial – where redundancies within each image frame of a video are independently removed (e.g. image compression) and, temporal – where redundancies across adjacent frames within a video are codependently removed (e.g. motion compensation). Codecs which do both forms of compression are ubiquitous, e.g. H.264 which is used in most modern FPV goggle DVRs. Video codec which do only spatial compression are less common, but one popular example used in cheap video surveillance and older FPV DVRs is MJPEG due to it being quite inexpensive to implement, however it produces a lower quality video with a higher bitrate (and consequently larger file size).

Another important detail worth considering about these two aspects of video compression when applied to FPV is the impact of irrecoverable transmission bit errors on the decoding of the video. The impact of an irrecoverable error in a spatially compressed frame really depends on the compression algorithms used, e.g. JPEGs in a MJPEG video use a combination of lossy quantised DCT compression blocks followed by lossless REL and entropy coding (e.g. Huffman coding) and bit errors within this data are likely to lead to the corruption of the entire frame. Bit errors in temporally compressed frames are even more detrimental and would likely lead to the corruption of an entire Group of Pictures (GOP) – more on this later – and several frames would have to be dropped.

Also when dealing with real-time video streaming applications like FPV it is worth noting the latency introduced by these two aspects of compression. Depending on the types of algorithms used in spatial compression the latency can vary anywhere from the time taken to encode an 8×8 DCT block of the image to (assuming entropy encoding is used as in JPEG) the time taken to DCT encode all blocks then entropy encode the entire image frame. If temporal compression is used as well, we then need to add in the time it takes find redundancies across two or more adjacent image frames and remove/re-encode them, so at least 2 frames of latency.

A hypothesis on HDZero’s Codec

Now this is purely speculation on my part, since as far as I know Divimath hasn’t published any details on how their codec works, but based on what we know about video compression and the observed behaviour of the HDZero system we can safely make two assumptions:

1. HDZero does not use temporal compression in its video codec.
2. The spatial compression in HDZero’s codec is not compressing the entire frame.

Contrary to popular opinion, the main reason for these assertions is not the low latency of the system, since “Zero latency” H.264 video codecs have been documented. Rather, it’s mainly because if either of these strategies were used, irrecoverable transmission bit errors would result in the loss/corruption of an entire image frame, which we know is inconsistent with the way HDZero handles interference. In fact, upon closer inspection HDZero video has what appears to be 8×8 pixel DCT block-level corruption:

A cropped and magnified shot of some HDZero breakup on one of my test flights, showing the corrupted 8×8 DCT block patterns.

Anyone can also visually confirm that these corrupted blocks appear to be badly decoded DCT coefficients which are normally 8×8 pixels in size:

I’ve actually counted the number of these blocks in a still frame of 720p DVR recording using the size of the corrupted ones as a guide and found there to be 128×96 blocks in a frame. For what I’m about to say next, we need to keep in mind that the 720p resolution of H.264 DVR recording happens after the video is decoded and bears little in resemblance to the original transmission format. So, if each block is indeed 8×8 pixels in size, then the original source image format appears to be a meager 1024×768 pixels which would suggest the original 720p image from the camera is downscaled and changed to a 4:3 aspect ratio before encoding, which results in 20% loss of horizontal resolution.

Working backwards from these assumptions, we can do some calculations to determine the data budget available for each of these 8×8 pixel DCT block, based on a 100Mbps channel bitrate:

=> bitrate per frame / total DCT blocks per a 1024x768 frame
=> (100e6 / 60) / ((1024x 768) / (8 x 8)) = ~135 bits (17 bytes)

For YUV420 image data, luminance (Y) is DCT encoded in full resolution while the two chroma components (UV) are DCT encoded at 1/4 resolution, so I suspect they would probably use 4×4 DCT blocks for U & V to align with the Y block boundaries. Therefore they’d need a way to compress an 8×8 (Y) and two 4×4 (U/V) DCT blocks (i.e. 96 bytes) into 17 bytes. This task does not seem so insurmountable after the DCT quantisation is applied, i.e. a process of scaling down the DCT coefficients so that high frequency detail components are reduced to zero, since our eyes care more about low frequency details than high ones. I suspect Divimath are only sending the top 10% or so of low-frequency coefficients for Y DCT block, and the top 20% for U/V DCT blocks. For Y, this would be the DC and 5 lowest frequency coefficients, and for U & V this would be the DC and 2 lowest frequency coefficients, for a total of 12 coefficients, and since these values are heavily quantised they can probably be encoding in smaller word lengths than 8-bit bytes, offering further compression.

An 8×8 DCT block before (left) and after (right) quantisation, with the DC and top 5 low frequency coefficients scanned out as -26, -3, 0, -3, -3, -6 for transmission.

Carl also mentioned that Forward Error Correction (FEC) wasn’t applied to all transmitted data but rather conditionally based on data importance, so I’d expect that the Y-DCT blocks would be sent with a 1/2 or 3/4 coding rate (50% to 25% overhead) while the UV-DCT blocks could be sent without any error correction coding.

I don’t deny that these numbers are all contrived, and no consideration was made for other data streams such as frame synchronisation and OSD telemetry from the flight controller to be multiplexed across the channel. But as far as formulating a hypothesis is concerned, I feel the approach is sound and would allow for irrecoverable transmission bit errors in a single UV DCT (or Y DCT) block to only ever result in the corruption of that block.

Resolution is not the same image quality

And this brings me to what I believe is the most important point I want to make concerning HDZero, which has been discussed very little in almost all the YouTube reviews of digital FPV systems I’ve seen thus far. And that is the focus on system resolution, without much mention of image quality i.e. how much detail is preserved in the image reproduction. Consider this chroma test pattern captured by HDZero:

A still frame of HDZero video recording a chroma test pattern.

The alternating red and blue bars should have straight edges, but you can clearly see the excessively quantised lower detail encoding of the U & V components creating a jagged square edge, which is quite a contrast to the straight gray scale (Y) edges elsewhere in the still. For reference, here is the original test pattern:

It’s worth noting that I captured this still frame from a DVR recording, which as discussed before is itself an H.264 re-encoding of the original decoded HDZero transmission video format, so even more detail is lost here, but notice how crisp and sharp the OSD font and icons are, since they appear to be overlaid in the VRX and only get encoded once to H.264.

I guess the point I’m trying to make is saying your system supports 720p doesn’t mean much if you’re downscaling and throwing away about 90% of the fine details. To be honest, I would much rather have 480p at even 50% quality. To justify this, consider the following video recordings of an EIA 1956 Resolution Chart which was traditionally used for testing analog TV systems, and notice how the 480p GoPro video at the top has far higher image quality despite having half the resolution of the following HDZero 720p DVR recording. If you freeze HDZero footage, you can even see the compression artifacts from the excessively quantised DCT block patterns showing. Lastly for fun, I’ve added a 480i analog DVR from a Runcam Nano3 800TVL camera at the bottom, and you’ll notice some of the numbers are far more readable for analog than HDZero.

I would love to see YouTube reviewer doing more of this kind of testing of digital (and even analogue) FPV systems going forward, maybe with a larger chart further away from the camera so the focus of the lens doesn’t affect the results as I felt they might have in these close up tests.

System Latency

Now we come to the matter of latency and this is where HDZero excels when compared to other digital FPV systems. But as with our previous discussion on resolution vs image quality, we need to be clear about what’s actually being measuring in the numerous systems latency tests which are available online. To break this down, let’s look at all the stages in the video feed where latency can be incurred, starting from the input:

Camera latency

As I see it, there are actually two sources of latency with digital video cameras, which influences what some reviewers have described as time to first-bright-pixel (TFBP) vs time to first-bright-frame (TFBF) in FPV latency tests:

TFBP determiner: The TFBB is largely influenced by the time taken from when the image sensor is read to when a horizontal scan line of pixels is outputted across the MIPI CSI-2 interface. If our 8×8 DCT theory holds true, then the DM5680 needs at least 8 scan lines of image data from the camera before it can start encoding the first row of DCT blocks. Now while Carl suggests that HDZero uses global shutter cameras, and this may have actually been the case for the original Byte Frost system which apparently had a Caddx Camera, all the newest cameras starting with the Shark Byte Runcam Nano HD up to the latest HDZero Micro/Nano V2 cams are specified as having rolling shutters and you can even see the tell tail artifacts such as “spatial aliasing” of the props and the “jello effect” in some DVR footage. So in the absence of a global shutter we can safely assume each frame of the image does not get buffered in the camera but rather scan lines are transferred immediately after they are captured and processed by the CSI-2 protocol stack. HDZero has actually documented their fixed low latency as being ~3ms (Δt₁ in their diagram) for TFBP when using their new HDZero goggles, which is quite impressive and means MIPI CSI-2 latency is probably somewhere around ~2ms, if we assume it matches that of analog cameras.

Now it’s worth noting that TFBB tests are normally conducted using light from an LED which when switched on will be captured by all pixels on the sensor simultaneous so the measure time is not influenced by the position of the scanline at that time, and a bright pixel is guaranteed to be captured somewhere in the frame.

TFBF determiner: The second source of latency is the camera frame rate, and is influenced by the time it takes for the camera to emit a full frame, i.e. to scan and emit all the bright/illuminated pixels in the frame during an LED latency test. For a 60fps camera this is 1/60 or 16.7ms, since this is the interval between frames, but also we need to include the TFBP latency which gives us ~19.7ms (i.e. Δt₂ in the diagram above).

Video encoding/decoding and RF channel latency

Again, using the HDZero goggles fixed low latency documentation as a guide, if the glass-to-glass TFBP latency is ~3ms and the camera latency is ~2ms then video encoding, decoding and the RF channel must introduce ~1ms of latency.

VRX HDMI and Goggle OLED driver latency

Thanks to Chris Rosser’s latency tests of the HDZero VRX we know that the system has a 12ms TFBP when paired with the Skyzone Sky04x goggles, so subtracting the ~3ms known glass-to-glass latency for the HDZero Goggles we see that somewhere between the HDMI transmitter and Sky04x OLED driver we’re getting an additional ~9ms of latency for 720p@60 video. I’ve read the IT66121FN datasheet and have not found any latency numbers or mentions of buffering which can add delay, but given than Sky04x OLED panels have been shown to have ~2ms TFBP for analog video, I suspect the extra latency comes from either the HDMI transmitter in the VRX or the HDMI receiver in the analog goggles. This would explain why HD FPV systems with both VRX and goggle options have their lowest latency when integrated directly into goggles. Surely Divimath was aware of this HDMI limitation when they embarked on making their own Goggles to completely eliminating the HDMI dependency.

Conclusion

Well this review ended up being much lengthier than I was planning but I’ll end with my final thoughts on the system. If you’re an FPV pilot who cares more about low latency than high definition image quality (e.g. racers) then I think HDZero is a worth while improvement over analog, especially with their new 90fps camera offering better latency than the best analog setup, but stay away from the VRX and it’s ~9ms of extra latency and get the HDZero Goggles instead. Also, HDZero seems to be quite popular amongst micro pilots due to the low weight of the whoop style VTXs.

While I don’t deny that HDZero image quality is far more consistent in colour reproduction than analog I feel the excessive compression goes too far and it ends up being only marginally better than analog due to the significant loss of image detail. Then there is matter of the digital breakup which I find is way more abrupt and distracting than analog breakup. For me as a freestyler who occasionally does a bit of proxy, I don’t think the latency benefits are as compelling as the higher image quality offered by other HD FPV systems. So while I’m still keeping my HDZero setup on a few quads, I’ve decided to give the new WalkSnail Avatar HD VRX a try to see if it delivers on image quality without compromising too much latency. I hope to write about that experience in my next blog post. Bye for now.

SimonK ESC firmware on Atmega88

I’ve recently started a project to build a quadcopter using discrete parts off Amazon, however it hasn’t quite been going according to plan. For starters, the four “El Cheapo” Mystery Cloud 30A electronic speed controllers (ESCs) I purchased were a epic failure after they would randomly turn off if throttled down too quickly.

Now these ‘puppies’ come with zero documentation – no surprises there given the price – but after quite a bit of digging around I managed to find a document here describing how to program them. Unfortunately this only served to confirm how basic these devices were, since I could find no option in there which would change the unwanted behaviour.

Further research seemed to indicate non-quadcopter ESCs generally expected PWM signals at a rate of about 50 Hz, and anything faster than this just gets averaged out by the firmware. My quadcopter uses a KKMultiCopter V5.5 flight controller board running the XCopter v2.9 XXControl Korean firmware from Minsoo Kim. It generates a 495 Hz PWM signal to the ESC – I was able to verify this with my scope – and they seem to be unable to cope with rapid changes in pulse width. Generally the solution to this is to replace the stock ESC firmware with SimonK firmware (assuming your ESC is supported) to get it to accept the 495 Hz PWM signal. So naturally this was my next move.

I checked out the device compatibility chart and found that my Mystery Cloud 30A was supported *screams and claps* by the Tower Pro build of the firmware (tp.hex). Luckily I got a USBasp in-system programming (ISP) adapter with my KKmulticopter board – which I had already successfully used to update the firmware – so I figured it was just a matter of following the SimonK instructions on how to strip open my ESC and reflash the firmware. But no, not quite so simple.
For starters, my ESCs did not come equipped with any pin-header connector for attaching my ISP adapter – why the bastards – but rather a 2 x 3 (6 wire) cluster of terminals could be clearly seen printed onto the board so first I needed to find a way of interfacing with terminals, and I wasn’t going to take temporarily soldered wires as an option. Several failed attempts, 1 eraser and 6 ball head pins later, I came up with the contraction shown here which had to be held in place by PCB holder so that it was perfectly positioned over the ISP pads of the ESC board while it sat snug in a bench vice.

So you can imagine my anguish after I was finally able to flash the SimonK firmware onto my ESCs, only to discover I that it wouldn’t work because my Mystery Cloud 30A board (probably as of a recent version 2) was fitted with the new and improved Atmega88 microcontroller, whereas SimonK firmware was only supported on Atmega8 based ESCs!!!

But all hope wasn’t lost yet. After spending a bit of time reading Atmel’s replacement guide to understand just how different these two chips were, I realised that it might not take that much effort to port the SimonK firmware to the Atmega88. A week later and I’ve finally done it and the ported code works fine on my Atmega88 based ESC. I’ve provided a diff patch here for anyone willing to try it (at their own risk of course). Admittedly, my current patch is quick and nasty. It breaks support for Atmega8 and since I haven’t gotten around to patching the bootloader code, I’ve also had to disable this for now. In the interest of contributing back to the SimonK project I’m planning to tidy up my changes at some point so that Atmega88 support just becomes a build configuration, and also port the bootloader code, and then submit these for integration into the SimonK project (if they want it).

The following steps briefly describe how to apply the patch and assume (for simplicity) the build is being done on a Ubuntu machine although all the tools used are available under other O/Ses and Linux distributions. So here’s what you’ll need to install if you don’t have it already:

sudo apt-get install git
sudo apt-get install avra
sudo apt-get install avrdude

Now let’s get the source, patch it and build the firmware binaries:

# clone the SimonK source repository
git clone https://github.com/sim-/tgy
cd tgy

# ensure we're patching the right revision/commit level
git reset --hard 2e13dfa

# get the patch
wget https://simonk-m88-patch.googlecode.com/git/tgy.asm.diff

# apply the patch
git apply --ignore-whitespace tgy.asm.diff

# get a local copy of m88def.inc from avra install
cp /usr/share/avra/m88def.inc .

# build the firmware *.hex files
make

Assuming that all went well you now need to flash the appropriate *.hex firmware file for your ESC. At this point it would be great if we could just use the KKMulticopter Flash Tool which is a user-friendly GUI wrapper around avrdude for choosing your programmer, firmware file and flashing your ESC, however it currently appears to only support flashing “Atmega 8-based brushless ESCs” which wouldn’t work for Atmega 88-based ESCs as the device signatures won’t match. So instead I had to just use avrdude directly from the commandline:

$ avrdude -p m88 -P usb -c usbasp -e -U flash:w:tp.hex:i

avrdude: warning: cannot set sck period. please check for usbasp firmware update.
avrdude: AVR device initialized and ready to accept instructions

Reading | ################################################## | 100% 0.01s

avrdude: Device signature = 0xffffff
avrdude: Yikes!  Invalid device signature.
avrdude: Expected signature for ATmega88 is 1E 93 0A
avrdude: erasing chip
avrdude: warning: cannot set sck period. please check for usbasp firmware update.
avrdude: reading input file "tp.hex"
avrdude: writing flash (8192 bytes):

Writing | ################################################## | 100% 2.52s

avrdude: 8192 bytes of flash written
avrdude: verifying flash memory against tp.hex:
avrdude: load data flash data from input file tp.hex:
avrdude: input file tp.hex contains 8192 bytes
avrdude: reading on-chip flash data:

Reading | ################################################## | 100% 2.05s

avrdude: verifying ...
avrdude: 8192 bytes of flash verified

avrdude: safemode: Fuses OK

avrdude done.  Thank you.

Not show above as a precaution is the “-F” override flag I needed to use since in my case the device signature wasn’t even preprogrammed into the Atmega88 of my Mystery Cloud 30A ESC (i.e. it was set to all F’s). However it’s best to try flashing your ESC without that flag set initially, because apparently incorrectly wiring your programmer to the ESC can also give a bad device signature.

SL4A Bluetooth Controller

Having had a bit of an unproductive Easter break I think it’s about time I got back to posting the concluding details of how the transmitter for the Bluetooth RC was implemented. I built the PIC-based motor and servo controller before acquiring the BlueSMiRF module so initially I wrote the following simple Python script based on the PyGame module to control the board via the direct (wired) serial connection using a USB gamepad.

#!python.exe

import pygame
import math
import serial
import threading

MAX_SERVO_POS = 0x3f
MAX_SPEED = 0x1f

def clamp(min_val, max_val, val):
	return max(min(val, max_val), min_val)

# initialise pygame module
pygame.init()
clock = pygame.time.Clock()
running = 1

# initialise serial port 8 [COM9] for direct serial connection
ser = serial.Serial(8, 19200, timeout=1)
# initialise serial port 5 [COM6] for Serial over Bluetooth
#ser = serial.Serial(5, 115200, timeout=1)

print("using serial port: %s" % ser.portstr)

# reset microchip
ser.setRTS()
pygame.time.wait(500)
ser.setRTS(False)

# setup asynchronous serial port reader
class AsyncSerialPortReader(threading.Thread):
	def run(self):
		while running:
			debug_output = ser.readline()
			if debug_output:
				try:
					print(">>%s" % debug_output.decode("utf-8"))
				except:
					continue

port_reader = AsyncSerialPortReader()
port_reader.start()

# initialise joystick
print("found %d joysticks\n" % pygame.joystick.get_count())
joystick = pygame.joystick.Joystick(0)
print("initialising joystick: %s\n" % joystick.get_name())
joystick.init()

# setup allowed events
pygame.event.set_allowed(None)
pygame.event.set_allowed(pygame.QUIT)
pygame.event.set_allowed(pygame.JOYAXISMOTION)

# enter joystick event loop
while running:
	for event in pygame.event.get():
		if event.type == pygame.QUIT:
			running = 0
		elif event.type == pygame.JOYAXISMOTION:
			if event.axis == 0:
				servo_pos = math.floor(MAX_SERVO_POS * (1 + clamp(-0.7, 0.7, -event.value))/2)
				print("setting servo position %s" % servo_pos)
				ser.write(bytes([servo_pos]))
			elif event.axis == 1:
				speed = math.floor(MAX_SPEED * clamp(-1, 1, event.value))
				dir = 0 if speed <= 0 else 1
				speed = abs(speed)
				speed_dir = 1<<6 | dir<<5 | speed
				print("setting motor speed and direction %s" % bin(speed_dir))
				ser.write(bytes([speed_dir]))
		pygame.time.wait(100)

	clock.tick(20)

# cleanup before exiting
port_reader.join()
ser.close()
pygame.quit()

The code basically bootstraps the PyGame library and enters into an event loop, processing joystick events on the gamepad by mapping them to control commands for the car and writing these to the serial port. The event loop is rate limited because the absence of flow-control on the serial connections means that it is quite easy to overwhelm the PIC UART code with data. The script also includes an asynchronous reader thread for printing diagnostic information outputted from the board. When the BlueSMiRF wireless serial modem was integrated into the board, it was just a matter of commenting out line 20 and using line 22 instead which establishes the serial connection over my PC’s Bluetooth adapter i.e. using the RFCOMM protocol. Note, the BlueSMiRF must first be paired to the PC (default PIN 1234) before the Serial Port Profile (SSP) COM port becomes available for use.

Device properties of BlueSMiRF (a.k.a FireFly-E52F) when paired with PC, showing RFCOMM service

The final step was to somehow port this code to my Android phone, using the accelerometer as if it were a free floating joystick. I decided on the Scripting Layer for Android (or SL4A)  platform which is a neat little collection of facades over the traditional Android Java API, with a wide variety of language bindings thus allowing one to quickly write simple applications in their favourite scripting language to harness the power of their Android device. Unfortunately the Android install packages for these apps are quite mainstream enough to be on the PlayStore so I had to manually download and install them from the website:

• sl4a_r6.apk – The base SL4A install package
• PythonForAndroid_r4.apk – The Python interpreter and standard libraries install package

Here is my admittedly crude port of the earlier code to SL4A. It uses the BluetoothFacade and SensorManagerFacade APIs:

import android
import threading
import math
import sys

MAX_SERVO_POS = 0x3f
AXIS_LIMIT = 4.5
MAX_SPEED = 0x1f

def clamp(min_val, max_val, val):
   return max(min(val, max_val), min_val)

# setup asynchronous serial port reader
class AsyncSerialPortReader(threading.Thread):
   def run(self):
      while running:
         print droid.bluetoothReadLine(connid).result

port_reader = AsyncSerialPortReader()
port_reader.start()

# initialise Android facade
droid = android.Android()
connid = None
running = True

# establish Bluetooth connection to SPP UUID
droid.bluetoothConnect('00001101-0000-1000-8000-00805F9B34FB')
conns = droid.bluetoothActiveConnections()
print conns

# failed to establish connection?
if not conns.result:
   sys.exit(0)

connid = conns.result.keys()[0]

# request rate limited accelerometer sensor data
droid.startSensingTimed(2, 200)

# sensor event loop
while running:
   sensordata = droid.eventWait()
   if isinstance(sensordata.result, dict):
      d = sensordata.result['data']

      servo_pos = int(math.floor(MAX_SERVO_POS/2 * (1 + \
         clamp(-AXIS_LIMIT, AXIS_LIMIT, d['xforce'])/AXIS_LIMIT)))
      droid.bluetoothWrite(chr(servo_pos), connid)

      speed = int(math.floor(MAX_SPEED * \
         clamp(-AXIS_LIMIT, AXIS_LIMIT, AXIS_LIMIT - d['zforce'])/AXIS_LIMIT))
      dir = 0 if speed <= 0 else 1
      speed = abs(speed)
      speed_dir = 1<<6 | dir<<5 | speed
      droid.bluetoothWrite(chr(speed_dir), connid)

 

DIY Bluetooth RC Car

Bluetooth RC car disassembled

Last year I started a project to convert my (then) four-year old’s busted Radio Controlled (RC) car into Bluetooth controlled one which could be driven using a smart phone or tablet. The original car, modeled after a “Fast and Furious” Nissan Silvia, was a pretty basic RC with 2-channel on-off (non-proportional) style controls, and while Micah enjoyed playing with it I thought it might be nice to pimp it out a bit and really make it a fun toy. Well actually, I thought at the very least it would be fun for me to try, even if I ended up making a bigger mess of it than it already was. It took a bit of work but I managed to pull it off, and while it had a couple of flaws, I thought it might be worth sharing my experiences for the benefit of any other hobbyist who might want to give this crazy experiment a try.

High-level diagram of the PIC based Bluetooth receiver

The concept was actually quite simple, nothing novel really and besides the Bluetooth interface the only other requirement was for the car to have proper proportional steering and acceleration because let’s face it, simple on-off controls are pretty boring. So this meant that I had to swap out the car’s simple DC motor steering mechanism and replace it with a proper micro-servo. Anyhow getting down to the specifics, at the heart of the device was an 8-bit PIC16F88 Microcontroller which was integrated with a  BlueSMiRF Bluetooth serial modem from Sparkfun using the PIC’s UART module. The BlueSMiRF is great because it implements the RFCOMM protocol which basically emulates a 3-wire asynchronous serial port with TTL level signaling over the Bluetooth channel, hence it is transparent to the PIC which just sees it as a serial I/O device from which it receives control instructions and to which it can write diagnostic information. For communicating with the PIC I used a simple protocol based on 1-byte (8-bit) commands described in the below table. Bit-7 is currently unused.

Instruction	CMD Protocol Bits								Notes
	7	6	5	4	3	2	1	0
Steer Left/Right	–	0	X	X	X	X	X	X	CMD<5:0> is a 6-bit heading with 00h being full left and 3fh being full right
Drive Forward	–	1	0	X	X	X	X	X	CMD<4:0> is a 5-bit speed with 00h being off and 1fh being full speed
Drive Backward	–	1	1	X	X	X	X	X

Non-shorting H-Bridge circuit diagram

The motor driver circuit is a design for a non-shorting H-bridge which uses a CMOS 2-4 line decoder, some discrete components, plus I’ve added diodes for back-EMF protection. At the time I didn’t have access to any monolithic H-bridge IC (e.g. the SN754410) so I borrowed this circuit since I figured if anything, the medium power transistors can handle up to 1.5A each and having four separate packages would provide better thermal dissipation than if they were lumped together in the same package. The speed (SPD) input of the H-bridge is driven by the PWM module of the PIC running at 10KHz, whilst the direction (DIR) input is just driven by a regular I/O port. At some point I’d like to add a feedback loop based on measurements of the motor’s back-EMF made via the PIC’s A/D converter, since this would allow for more precise motor speed control.

The servo is pulse input is driven by a software PWM signal generated using timer preset functionality of the Timer1 module along with overflow interrupts which do the actual bit-bashing out an I/O port. I suspect it might even be possible to multiplex control of up to 7 servos (using other I/0 ports) within this software routine, given that each pulse has a maximum length of 2.7ms and are spaced at 20ms apart.

Digital scope trace of motor voltage drop during spin-up while cycling through forward and backward drive

My understanding is that normally for robotics projects, digital devices (e.g. PIC controllers) are run off a separate power supply (or in the case of battery-powered ones, different cells) from noisy analog devices, motors being the worst of these. However, because I intended on leveraging the existing 4 x AA battery compartment in the car, this wasn’t going to be an option for me, since I would need at least an additional 2 x AA cells just for the digital circuits. Now the PIC 16F88 required at least 4V and I wanted to use 1.2V nickel-cadmium rechargeable cells, which meant the car would have to work with 4.8V minimum. So at first I thought I could get away with connecting the motors and digital circuits to the one common 4.8V supply and just regulate and filter power going to the PIC, but as it turned out the DC motor consumed large start-up and stall currents resulting in as much as 1V dips in the supply voltage and this caused the digital circuits to brownout.

A 4.8V to 5V regulated voltage booster based on the MAX660 and LM2936-5.0 ICs

I came up with a solution using a MAX660 positive voltage doubler IC to boost the 4.8V supply to 8V, followed by a 5V low-dropout (LDO) regulator. At the input to the doubler I also added a 1000uF electolytic capacitor to act as a reservoir, which is charged via a diode to prevent it from discharging back out to the motor during voltage drops.

Scope trace comparing voltage doubler and LDO regulator outputs during supply voltage drops due to DC motor spin-up

As you can see from the new scope traces, the final out still has the occasional dip but at least it’s still within the 4V operational range of the PIC as well as the other CMOS ICs.

 

 

 

Tiny Bootloader serial programmer adapter

Now for some details on how the PIC firmware was deployed. The 16F88 was initially flashed with the Tiny Bootloader firmware using a KIT-149 PIC programmer I bought a couple of years back when I first had the crazy idea to get back into hobbyist electronics. Tiny Bootloader is great because it allows the main program firmware to be loaded to the PIC while it’s in-circuit via the UART interface, and best of all the bootloader has a “tiny” memory footprint. I included a PGM (program) connector on the board to which I could connect an RS-232-to-TTL level converter adapter (documented on the Tiny Bootloader site) and using the Tiny Bootloader desktop utility I was able to flash new revisions of my firmware on to the PIC via the serial port on my PC. My adapter also included a circuit for resetting the PIC via its MCLR pin which allows the desktop utility to invoke the bootloader by performing a hardware reset and was powered through a two wire connector on the board.

For anyone who’s interested, the PIC source code is publicly available on Google Code. Just grab a clone of the repository using mercurial:

hg clone https://krisdover@code.google.com/p/pic-bluetooth-rc-car-receiver/

The code was written in C and developed in MPLAB X IDE v1.60. In my next post I’ll cover the details of the controller for driving the car.

Finally I thought it best to end with some videos showing the end product in action:

PCB layouts using CAD

Now the PCB kit I used described how I could draw my circuit layouts directly unto the copper clad board using an special pen with etch-resistant ink, however lacking the dexterity of an artist I chose to get myself some CAD software for generating PCB layouts. After a bit of research I decided to go with EAGLE from CadSoft which has free lite edition, mostly because it seemed popular. I have to say that as a novice user of any PCB layout editor, my first impression was that I found the user interface very unintuitive, but thanks to the many user tutorials on YouTube I was able to get enough of a grasp of the tool to bend it to do my will, although admits occasional outbursts of frustration.

TCM8230MD EAGLE device symbol, package and a superimposed image of the actual device contacts

I was able to determine than the CMOS camera’s package was a QFN-20 but a search online did not yield any EAGLE libraries for this device, so the challenge for me lay in creating one which matched the CMOS camera’s footprint. Turns out this is not too difficult, especially if you’re in possession of the annotated device drawings, then it just a matter of reproducing the footprint in CAD with matching dimensions. I found that just printing the package layout on a plain sheet of paper and superimposing the actual device over it was a great way of verifying the measurements. The library is available here: tcm8230md.zip for download for whoever might be interested.

Camera soldered to breakout board

Here’s what the final mounted device looks like. Solder paste actually makes working with SMD easier than I thought it would be. I failed to mention it before, but I had to check for any unexpected short circuits with a multi-meter before and after soldering the device to the breakout board. These kinds of defects could be caused by poor PCB construction or just excess solder.

 

Now it’s just a matter of setting up the PIC microcontroller and wiring this baby up.

Print, iron, etch, repeat

A200 Etch-set

So last time I mentioned I had grabbed a beginner’s etch set – the A200 by a German company Kemo Electronics – from JayCar with ambitious plans to make a breakout board for this surface mounted CMOS camera sensor I wanted to play with. In retrospect, I could have purchased most of the items in the kit individually and I would have probably gotten better value for money that way, but I guess where this kit shines is that it allows beginners to inexpensively experiment with two methods of PCB development by including most of what you need in one package. Now one of these methods – involving the use of a photoresist coated board – required the use of UV lighting and a photo-positive template printed on transparent media, which I thought was a bit too sophisticated for my first attempt so I opted for the simpler toner transfer method.

Now my inkjet printer wasn’t quite going to do the trick here, so I grabbed a used HL-2140 Brother laser printer off Trade Me for $25 (great deal). The OEM replacement toner cartridges for these printers tend to be overpriced, hence why people probably just prefer to sell them and buy a new one at a fraction of the price it would cost to restock the old one. The trick is to find a used printer for which third-party compatible replacement toners are available since these tend to be much cheaper. I got one for $18 off Trade Me, which is about 1/5 of the price of an OEM one.

circuit board with toner transferred using magazine page

Next I needed a print media which would facilitate the toner transfer process. My research online seemed to point to glossy photo paper as being best, however the 20 sheet pack of Brother BP61GLA A4 paper I tried was an endless source of frustration. Ironing this paper with the PCB prints unto the copper clad board would not only transfer the toner, but also glossy substrate which seemed to have plastic consistency. This left an etch resistant semitransparent plastic film over the board which had to be delicately scrubbed off, taking care not to ruin the desired transferred toner pattern. After repeated making a mess of this, I followed some other advice online and used magazine pages instead, which are a bit thinner so if your printer has a “paper thickness” setting like mine, set it to “thin paper” to save yourself a lot of stress with paper jams. Magazine pages are wax based and this means that they won’t bond strongly to the toner, and with a little heat from the iron, readily transfer the patterns to the copper clad boards. They also required a lot less soaking in warm water to cleanly separate from the board than the glossy photo paper. The only caveat was that I just had to be careful not to apply too much pressure with the iron while transferring the pattern, since this caused the molten toner to bleed laterally and in some cases form bridges between traces. I then used a permanent market to touch-up any imperfections in the transferred circuit. One important lesson I learnt was how important it was to properly clean copper surface of the board in preparation for toner transfer otherwise any contaminants (e.g. I’m guessing fatty acids from hands) would cause the transferred toner not to bound with copper board in some spots. I found isopropyl alcohol on a paper towel (not dish washing detergent as some suggested) works best for this.

etching PCB in sodium persulphate solution

Contrary to the instructions from the kit, I did not dissolve the entire 100g sachet of sodium persulphate (the etchant) into 1/2 a liter of water but rather I only used 1/4 of the sachet (weighed to 25g) with 1/2 a cup of warm water since I only planned on etching very small board. This stuff is toxic and requires special disposal so it’s best to just keep it in a container reuse for as long as possible by adding addition etchant when necessary. I just needed a hot water bath to raise the temperature of the solution for subsequent reuses. My board etched within 1 hour and from there it was just a matter of washing it clean with water, drying it, and removing the toner with acetone (nail polish remover) to reveal the copper traces.

Fun with PCBs

Last year I bought this Etch-Set For Electronic Circuits from Jaycar with the intention of having a go at making my own printed circuit boards (PCBs). I’d had lots of experience working with those experimenter’s Stripboards and Perfboards, which had been more than sufficient for most of the fiddling I did in the past, however I was always intrigued by the idea of rolling my own PCBs although I lacked the resources back then to make it happen.

A recent prototype using Stripboard

After acquiring the kit, the project I had in mind for the PCB – a PIC-based Bluetooth controlled car (more on this later) – ended up being implemented using Stripboard since in a sense it was just a proof-of-concept and these prototyping boards are great at allowing you to explore an idea without having a concrete design in mind. If you don’t mind soldering and you hate the restrictiveness of a solderless breadboard then Stripboards are perfect. You can pretty much place components where ever you like knowing that you can easily run jumpers to make connections or where necessary, create discontinuities in the parallel strips of copper with a small drill bit. I have found that with these boards, how condense the components can be laid out  is only limited by one’s imagination.

 

TCM8230MD CMOS VGA Camera Sensor

Anyhow, two months ago I finally decided to start my voyage into the wonderful world of DIY PCB fabrication, mostly because the project I had in mind could be implemented in no other way. Last April I had bought a CMOS VGA camera sensor off SparkFun with the crazy idea of incorporating it into my Bluetooth car. What I was really after was the JPEG version which was already conveniently mounted on a PCB with a TTL serial interface, however these were out-of-stock at the time so I thought I’d be a bit ambitious and try this cheaper alternative. So, knowing the only way I could make use of this surface mounted device (SMD) was to first solder it to a breakout board I set out to design and fabricate my very first PCB from scratch.

TCM8230MD breakout board

Now this sensor had been around for a while and quite a few people have already been through the process of creating breakout boards for it, for example this dude here. However I was determined to do this the hard way and design my own using one of the freely available PCB CAD software online. To make things even more interesting, it turned out that no package library with a component matching the device’s footprint was available online from Toshiba (the device manufacturer) so for starters I would have create that from the device’s form-factor drawings in the specifications documentation. As it turned out, I did get there in the end, and with a great deal of effort and trial and error I managed to create the simple breakout board on the right. To be honest, I was so pleased with the result that I was hesitant to mount the device for fear of making a mess of it. I thought maybe I could just frame it and keep it for motivation. 🙂

I’ve provided a link to the Eagle files here for anyone who might find them useful. In my next post I’ll go through the steps I followed to create this, my first PCB.

Why start a blog?

I had considered it several times in the past, but I could always find good reasons not to start a blog:

• What would I share? Okay fine, sometimes (on very rare occasions) interesting stuff does happen, but I’m a private person and I’d rather just keep it to myself.
• Maybe I’ve deceived myself! I’ve got nothing of value to share that people would be interested in reading. Besides, chances are someone has already had the same experience or idea  and shared it in a more meaningful way than I ever could.
• Even worse, what if the things I thought were worth sharing weren’t particularly relevant at all? I’m a strong believer in saying nothing at all if you’ve got nothing good to say.
• Fine, I can pretend it’s like keeping a not so private journal. But then it probably needs to at least be presentable and can’t stand the pressure of having to produce something of literary integrity.
• And even it did start it, I would need to make regular submissions right? Then I would procrastinate, and it would probably just end up being another one of my unfinished projects.

So naturally, armed with so many good excuses I had resisted that urge up until now. That is, until I recently gave serious consideration to some personal projects I have been working on and why it might be worth sharing them, when I was struck by the most compelling reason of all to start a blog:

What if someone is waiting to be inspired by an idea or story I have but haven’t shared?

I realized  that I had spent most of my time on the internet consuming the contributions of the online community selfishly, without considering that I myself might be able to contribute to others. After all being able to convey abstract ideas (mostly but not exclusively through language) is what sets up apart as humans and is a gift we should never despise. And what if all my original arguments were all wrong. Maybe an idea or a story doesn’t need to be original and the real value is in the way we as individual interpret and express it and even the way it might be reinterpreted thereafter. Furthermore, maybe I shouldn’t be too hasty in trying to decide my contribution’s relevance, but rather just be open to criticism and have it be decided by the online community (trolls and all).

Okay that’s all for now, I’ve already spent too much time on my first post 🙂