Raspberry Pi Project Page

Last Updated on January 14 2018

G-bot 4k (My affordable Pi3 data logger)

I've been looking to build a Pi data logger for some time. And the latest Pi3 Model B, with its quad core processor, at around £30 is the ideal tool.

So how do we go about building a data logger?

We need a computer, an electronic interface to allow us to read analogue voltages, a gps device to provide speed and location, and some software to process the information we'll be collecting.

Want to know more? Then read on....

All race car data loggers are GPS based, that is, they get the vehicle's position using GPS satellite data, to record the speed, direction, position, elevation, and the time & date. So we need a means of providing GPS data to the Pi3. Now there are a multitude of GPS capable HAT (Hardware Attached on Top) boards available, and the one I opted for provides not only GPS, but also has GSM/GPRS for mobile communications, battery backup, and has a lower power consumption, which is important if we decide to run the Pi from a mobile phone battery. (The Pi3 takes its power from a USB cable)

The next thing to decide is the programming language that we will be using to interface to the hardware. For now, for ease of use, I'll be working with Python. I would like to run Windows IOT on the Pi, and code in Dev Studio, but having looked at the interface, it seems way too complicated for a start, and Python is very versatile, but may not execute fast enough to allow me to capture the data at the speed that I want, so we'll start with Python, and see how it goes.

Anyone old enough (like me) to remember playing with Modems to connect to the internet, are probably familiar with AT codes. These were the commands that were used to initialise the Modem, select the Baud rate, and the phone number to dial. Well, interestingly, things havent moved on much, and the chipset on the HAT is programmed using AT codes. This is good news for noobies, as it makes setting up the HAT far simpler, as the AT codes are well documented, and it means you dont need to bother with internal programming of the chips. The AT commands effectively provide a nice easy user interface to the chips. eg The AT codes are interpreted by the chips and they do all the complex stuff for you.

So lets make a start

Once you have assembled the hardware (Pi, microSD card, 2.5A USB power supply, Screen, Keyboard, Mouse and the HAT module), the first step is to build the operating system that the Pi will load when it is powered up. I started off by downloading the Raspbian Stretch image file (2017-11-29-raspbian-stretch.img) from the Raspbian website. Once you have the file, and you have found a 8GB or greater microSD card, you will need to wipe the microSD card, and then use Etcher to write the downloaded and unzipped.IMG image file to the card.

Once this is done, the microSD card is plugged in to the Pi3, and the keyboard, mouse, HDMI (Video) and 2.5A USB power supply are attached, power is turned on, and the Raspbian interface appears on the connected HDMI screen. You'll get a tiny orange light on the Pi to signal that the power is present.

For the hardware, I was looking for a GPS HAT module for some time. Then I found the Waveshare GSM/GPRS/GNSS/Bluetooth HAT Expansion Board with Low-Power Consumption Based on SIM868, which not only had GPS, but also supports the latest GLONASS satellite network, that provides more accuracy than GPS alone. Then I read that the HAT also has a SIM card slot, that allows it to send and receive phone calls and text messages. Plus it has Bluetooth (which the Pi3 already has). And it came with the 2 metre long GPS antenna cable, and a few other bits and pieces, so I ordered one from Amazon and a few weeks later it turned up.

The Waveshare HAT device is so versatile that it can be even be programmed from a Windows Desktop PC, via the USB cable, so it doesnt even need to be used with a Pi. In my case, however, that was the intention, so with the Pi powered off, I carefully lined up the header GPIO (General Purpose Input Output) DIL pins, and pushed the HAT board down on to the Pi. The Waveshare HAT gets its power from the GPIO header pins, and passes the serial data directly to the Pi's UART, so there were no other connections to be made to the Pi. Plugging the HAT's GPS header cable in for the external GPS antenna was a bit fiddly, but once done, I plugged in the rest of the antenna to the header cable, and powered the Pi on to see what would happen.

Note: The Pi will need internet access to complete the next task, so setup the Pi3 wifi to connect to your home WiFi, or plug in a LAN cable to provide internet access.

This is where things didnt go quite according to plan. The instructions supplied with the HAT said that I had to turn off the UART for TTY access, and allow the HAT to send its data to the UART. So I manually edited /boot/config.txt and added the line enable_uart=1. After rebooting the Pi, to allow the change to take effect, the next step was to run minicom to establish a terminal session to the HAT. However, Raspbian doesnt come with minicom installed, so you need to run
 sudo apt-get install minicom
After several minutes, the minicom package is fetched, unpacked, and installed in to the Raspbian Pi operating system, and so now you can run minicom.

Open a command shell, and run
 pi@raspberrypi:~ $ sudo minicom -D /dev/ttyS0
A terminal session will open, and you should now be able to type in an AT command, and the HAT should reply with an OK. But in my case it wouldnt work. All I ended up with was gobbledygook on the screen from the HAT, and eventually it would just lock up. After lots of head scratching, I realised that the text that was appearing on the terminal session was a Login: prompt. This suggested that the Pi was still using the UART to allow a terminal session, which is not what we wanted. After doing some research, I read that the config program must be run to turn off the console session, which I thought should be the cure to the problem.

So I ran
 pi@raspberrypi:~ $ sudo raspi-config
and Select option 5, Interfacing options, then option P6, Serial, and select No. Then turn off login shell over serial, so that the serial login shell was disabled, and the serial interface is enabled.

After a reboot, I then once again ran
 pi@raspberrypi:~ $ sudo minicom -D /dev/ttyS0
and this time, when I typed AT and pressed Return, it was responded to with an OK, and I could then start to send codes to the HAT to switch on the GPS module and see what kind of data I could get back.

Now to switch the GPS electronics on
After getting familiar with the AT codes used to program the SIM868 GPS chip on the HAT, I realised that it would only update the GPS data at their advertised rate of 1Hz (once a second). Hmm, that wasnt what I expected. I wondered if there might be a way to force the SIM868 to provide updates at a higher rate, when I noticed that the Waveshare documentation included a section where you could send PMTK codes directly to the chip. Hmm, PMTK, that was worth a google. I then discovered that PMTK was the protocol used to configure a specific manufacturer's chip, but surely as the documentation mentioned int, the SIM868 must support it? So I found a web page that described how to change the update frequency, and sure enough, when I sent the PMTK code to the SIM868, it started updating at 5Hz. Success.
The next thing was to see if we could push the chip any further. Would it support 10Hz? So I sent it the PMTK command to update 10 times a second.
Voila, the time stamp being shown from the output data, was going up in 0.1s. Brilliant. This meant that the GPS HAT would provide data at 10Hz, which is what was needed for more accurate data capture. 1Hz wouldnt have been a high enough update speed, imagine how far the car would travel at 150MPH in just 1 second. I did then try even higher update frequencies, but the SIM868 chip just replied with an OK, and the timestamp didnt increase, so it's definitely topped out at 10Hz.

This animated GIF shows the Waveshare HAT, running at 10Hz, and the green LED flashes on and off to show it sending the data through the UART to the Pi.

Coding with Python

So the next step was to put a Python script together, to send the startup codes to the HAT board, rather than use a minicom terminal session, and I had a go at writing my first Python script. What became clear, was that the HAT needed a certain amount of time between commands sent to it, for it to process them, and after a bit of experimentation, I managed to come up with a working script that initialised the HAT's GPS module, programmed it to send updates at 10 times a second (10Hz) and capture the data being returned, which in turn was saved to a text file.

NMEA Output

Every update from the GPS chip, involves information output in what are known as sentences. Each sentence encodes specific pieces of information seperated by commas, which has a different meaning based on the first 5 (or so) characters. The exact specification is proprietary, but there is a particularly good online reference if you're curious. Luckily, the pynmea2 library takes care of much of this for us. There is a python library already written for interpreting the NMEA sentences sent from the GPS HAT, so you will need to install this and include the library in the Python, and experiment with reading the data back and making sense of it.

If you dont already have pip installed, do the following (pip is a package management system used to install and manage software packages written in Python)
 pi@raspberrypi:~ $ sudo apt-get install python-pip
To install the NMEA library for Python, run
 pi@raspberrypi:~ $ sudo pip install pynmea2
To collect the data, and use the Python NMEA library to parse it, the following Python code can be used:

The code above finds the sentence containing the GGA data, and then prints the parsed information to the screen. GGA is used for the essential fix infornation, so it includes the position information. The next improvement is to parse the VTG sentence as that contains the actual vehicle speed in kph.

NMEA Sentences

GGA - essential fix data
GSA - accuracy of the satellite fix, number of satellites being used, DOP (Dilution Of Precision)
RMC - NMEA has its own version of essential gps pvt (position velocity time) data.
VTG - Velocity made good. Ground speed knots, ground speed kph
GSV - Detailed satellite data, ie # of satellites in view

An example of captured GPS Data: [coordinates have been obfuscated]
Every sentence ends with a checksum value (* followed by number), to verify the integrity of the information in the sentence.

Some sentences are starting with GP and others with GL. The GL sentences are based on whats known as LORAN Data (LORAN is short for LOng RAnge Navigation). We are only interested in the sentences starting with GP.

Lets look at VTG in more detail:
This shows: $GPVTG, cogt, T, cogm, M, sog, N, kph, K, K*cs
	12.10	course over ground true (cogt)
	T	True (fixed field)
	[null]	course over ground magnetic (cogm) not output (empty)
	M	Magnetic (fixed field)
	0.18	Speed over ground in knots (sog)
	N	(fixed field)
	0.33	Speed (kph)
	K	kilometres per hour (fixed field)
	A*18	checksum
	<CR><LF>End of message
So the VTG sentence contains the actual vehicle speed, kph, so this sentence needs parsing by the Python program to pull that key piece of information out, therefore we shall ignore speed over ground in knots (sog)

And now look at the RMC data in more detail
This shows:
	122752.000	Fix taken at 12:27:52.000 UTC
	A		Status A=Active or V=Void
	5999.155767,N	Latitude 59 deg 99.155767' N
	00999.797128,W	Longitude 00 deg, 999.797128' W
	0.18		Speed over ground in knots
	12.10		Track angle in degrees True
	301217		Date - 30th December 2017
What we need to do is concatenate the relevant information to a single string, and write it to a file. The file could be closed automatically when the speed is <1kph, and then a new file is created when the speed goes >1kph. This way we would end up with small files, but maybe the thing to do is to only list the files that have more than say 5s of data in them, to save the user having to discard them.

Before we get started though, to reduce the amount of serial traffic, lets turn off the sentences that we're not interested in. The following PMTK command tells the SIM868 chipset to only output RMC, GGA and VTG sentences.
So I now have the Pi parsing the GGA and VTG data, and writing them to a text file, with a time stamp for the filename.

eg It writes:
G-Bot4k gps data capture test at 10Hz

Time, gpsLat, gpsLong, gpsAltitude, gpsSpeed(kph), gpsCourse
17:17:24.0, 5999.149370, 00999.785770, 113.228, 1.9, 215.39
17:17:24.2, 5999.149355, 00999.785787, 113.514, 1.13, 222.62
17:17:24.3, 5999.149353, 00999.785774, 113.602, 0.82, 222.62
17:17:24.4, 5999.149452, 00999.785767, 113.733, 0.44, 222.62
17:17:24.5, 5999.149582, 00999.785794, 113.542, 0.48, 222.62
17:17:24.6, 5999.149706, 00999.785839, 113.131, 0.4, 222.62
So the next step was to make the program wait for the Satellite Fix to take place, to avoid writing zeros to the file, and this is determined by the gpsQual flag. While the value returned 0, no fix has taken place, and there is no data to process. As soon as the flag changes to 1 or higher, the program starts processing the NMEA data, and writing to file. It can take a very long time (Minutes) to get a satellite fix when starting from a cold start. I will need to work out how to avoid cold starting the GPS circuitry every time.

KML output

The logical thing to do was to have the Python script create a second file, a KML file, that allows the track data to be read in to Google Maps, to overlay the course on top of the map, to show the vehicles path. This took a bit of digging, as the KML/XML file format isnt that well documented, but after a while I managed to get the file format right, and Google Maps will now let me import the KML file direct from the Pi, and it then shows the vehicles course.

Developing the software

The software is being developed under Idle Python 2.7.14, and is now up to the version shown below. I am working on a new control loop, to allow the program to close open files when the vehicle speed is zero for more than 5seconds, and a few other options will also be available.
#v7  wait for hat board to be turned on, then commence logging
#v8  when hat board turned off, close the open files and exit program.
#    Log raw nmea data to dbg file.
#    Send nmea sentence string just before capture begins, to turn on vtg,gga,rmc (was being ignored)
#v9  Removed illegal colon chars from generated filename to allow files to be copied using Explorer
#    Epoch calculation error fixed
#    Created write_kml_header routine
#v10 Shutdown Pi and power off when HAT turned off
#v11 Fixed gga.timestamp string length
#    Added number of satellites to output file gps.num_sats
#v12 I2C interface, read analogue voltages from the AB Electronics Pi ADC

ADC Convertor

For this project to progress, I've added an ADC (Analog to Digital Convertor) HAT board, stacked beneath the GPS HAT, on the same GPIO header connector. This allows the Pi to capture the analogue input voltages, from the four linear potentiometers that I've fitted to the shock absorbers on the car. This data will need to be time stamped, and saved to the Pi's disk, and after a run, the intention is to get the Pi to email me the file automatically to my gMail account, allowing me to compare the G-bot data alongside the data captured by the car's Life ECU, which is also time stamped using GPS.

We need to enable I2C on the Pi for the ADC HAT to work. To do that, run the raspi config utility, and enable I2C
 pi@raspberrypi:~ $ sudo raspi-config
Once that is done, run the following command to make sure that the I2C libraries are installed (They should be)
lsmod | grep i2c
If you can see items starting with i2c, you can then run the i2cdetect command to see the i2c card address.
sudo i2cdetect -y 1
You should see the following
     0  1  2  3  4  5  6  7  8  9  a  b  c  d  e  f
00:          -- -- -- -- -- -- -- -- -- -- -- -- --	
10: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
20: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
30: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
40: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
50: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
60: -- -- -- -- -- -- -- -- 68 69 -- -- -- -- -- --
70: -- -- -- -- -- -- -- -- 
Once the card is shown to be working, you can then install the libraries for the ADC card, and the code can be developed to extract the data from it.

We are also considering sending the data to the cloud, as the car is being driven, to give real time information on the track position and performance of the car. With the Pi, and a good phone signal, pretty much anything is possible.

www.g-bot4k.com is live

I'm going to put a couple of web pages together, to promote the project. The domain name is now live, and I've a landing page. So far so good.
Click here to visit www.g-bot4k.com


AT Commands
Raspbian Stretch
Waveshare GSM/GPRS/GNSS/Bluetooth HAT Expansion Board with Low-Power Consumption Based on SIM868
GLONASS satellite network
PMTK Codes
GPS reference
Python pynmea2 library