EnkiThe whitestar project is my attempt at building a reliable and fully automatic wardriving equipment for permanent installation in my car. It's used to constantly gather 802.11 data for research purposes and contribution to the https://wigle.net crowdsourced WLAN map.
This may not be the most recent project information page. I have copied this information from my private site: http://pop.fsck.pl/projects/whitestar/ and it's a good idea to check there for updates. I will try to post them here as well but without some automated system that I'm yet to build there can be a delay ;).
1 The hardware
The whitestar is built around a kektop (NTT HOME W 300P net-top style PC) with an Intel Atom 230 CPU (1.6 GHz), 1 GB of RAM and a small 16GB internal SSD. The kektop has two wireless cards and a GPS connected via USB for wardriving purposes. There is also an ELM327 interface connecting to the car OBD-II port. Additionally, I figured I should store all of the data outside of the internal SSD to increase its life expectancy. For this purpose I connected a 2.5'' 500 GB USB hard disk. A laptop HDD was chosen because of better vibration resistance making it possible for it to actually survive working in a car for extended periods of time.
2 Power
In the beginning the whitestar was powered directly from the car lighter socket via a step-up converter from Aliexpress (XL6009-based). One of the improvements I wanted to make after using it for a while was to add some basic UPS capability to it. This is because the cigarette lighter power source has some drawbacks in a car. There are some situations where the voltage there drops causing the computer to reboot loosing packet capture ability for the amount of time it takes to boot up again. Also, short stops in your trip (for example gas stations) should not cause the packet capture to stop and restart.
So there are two problems here. The first one is a momentary drop in voltage as the engine is started (which reboots the kektop). The second problem are short stops (think gas station) where it would be nice to keep the kektop running for a few minutes. To solve both of those I acquired a suitable battery. In the spirit of efficiency I simply purchased a powerbank with a 19 V laptop output that can supply enough power to run the whitestar for a few minutes. You would find it hard to be more generic than this brand:
During some initial attempts at connecting all of this together I figured out that I can't just plug the powerbank to car 12 V to keep it constantly charging and power the whitestar from it's 19 V output. This is because the charger built into the powerbank will not be able to keep up and keep the Li-ION battery charged. A different approach is therefore needed, where a relay will switch the load from car 12V to the battery and vice versa. To keep the kektop running during this time I figured I would need a supercap bank. Therefore I bought and built it:
The resulting capacitor bank has 10F@27V (10 x 10F@2.7V). It can power the kektop for a good few seconds. So I decided to build a first version of the UPS to test it out, without the battery and relays first. Just the supercaps so that I would see will the kektop survive an engine start without a reboot.
For now the connections are:
Cat 12V input <-> 10F@27V supercaps <-> SMPS <-> 19V <-> kektop
2 Software setup
The software that I used was based on a clean Xubuntu installation with a GUI. After that I added some packages and changed some configuration options.
All of the scripts mentioned are available on github: https://github.com/mgrela/whitestar
Below is a step-by-step guide for installing everything:
2.1 BIOS setup
First we need to enter the BIOS of the machine and set up two important options:
- enable automatic poweron on AC failure
- disable all other booting mechanisms except internal HDD
2.2 OS installation
The OS installed on the device is Xubuntu 15.10:
$ lsb_release -a No LSB modules are available. Distributor ID: Ubuntu Description: Ubuntu 15.10 Release: 15.10 Codename: wily
2.3 Disable console blanking and system splash screen
Console blanking can be a nuisance when you connect a monitor to the PC to watch the kismet monitor. Also, when the PC boots there usually won't be a monitor attached anyway so we can get rid of the splashscreen. We can do both of these things by editing the grub configuration in /etc/default/grub
$ cat /etc/default/grub | grep CMDLINE_LINUX_DEFAULT #GRUB_CMDLINE_LINUX_DEFAULT="quiet splash" GRUB_CMDLINE_LINUX_DEFAULT="quiet consoleblank=0"
We remove the 'splash' option and add a 'consoleblank=0' to disable console blanking. Don't forget to run 'update-grub' to generate the config files in /boot/grub.
2.4 Change default systemd target
The default systemd target for the Xubuntu installation is graphical.target. This will start the X server and launch a graphical login screen. We do not want that so we need to change the default target that systemd tries to reach when it boots.
# systemctl set-default multi-user.target
The multi-user.target provides us with an old-skool Linux text-based login.
2.5 Setup udev rules
We need to setup some udev rules so that the devices which we will use in configuration of other parts of whitestar will have some defined names. For example, the GPS unit should be always named /dev/gps, the OBD-II device should be always called /dev/odb and so on. Also, the network devices used for 802.11 scanning should have static kismet UUID identifiers attached. These files are put in /etc/udev/rules.d
2.5.1 /etc/udev/rules.d/30-whitestar-devices.rules
# WLAN cards
SUBSYSTEM=="net", ACTION=="add", ATTR{address}=="00:c0:ca:57:10:89", ATTR{type}=="1", NAME="alfa", ENV{KISMET_UUID}="39ed09aa-2dcd-4eab-b460-781de88f79d6"
SUBSYSTEM=="net", ACTION=="add", ATTR{address}=="00:15:6d:84:1a:78", ATTR{type}=="1", NAME="sr71", ENV{KISMET_UUID}="e8d964d0-9409-408f-a1d7-01e841bae7ed"
SUBSYSTEM=="net", ACTION=="add", ATTR{address}=="80:1f:02:8f:77:6c", ATTR{type}=="1", NAME="chibi", ENV{KISMET_UUID}="fb187219-afd4-4be8-871a-220d16fb5cb0"
# GPS receiver
SUBSYSTEM=="tty", ATTRS{idVendor}=="10c4", ATTRS{idProduct}=="ea60", ATTRS{serial}=="0001", SYMLINK+="gps"
# OBD-II cable
SUBSYSTEM=="tty", ATTRS{idVendor}=="0403", ATTRS{idProduct}=="6001", ATTRS{serial}=="A6008qzE", SYMLINK+="obd"
The first three rules set up a KISMET_UUID property for each device we want to use for packet capture. We match the card MAC address as well as its type to prevent the monitor devices created by kismet from running the rule a second time. Next rules setup static symlinks for the GPS receiver and OBD-II interface.
2.5.2 /etc/udev/rules.d/50-kismet.rules
SUBSYSTEM=="net", ACTION=="add", ENV{KISMET_UUID}=="?*", RUN+="/usr/local/bin/kismet-add-source $name:uuid=$env{KISMET_UUID}"
SUBSYSTEM=="net", ACTION=="remove", ENV{KISMET_UUID}=="?*", RUN+="/usr/local/bin/kismet-remove-source $env{KISMET_UUID}"
Here we specify two scripts that will be run when the previously selected network devices appear or disappear (for example when the USB cable disconnects by accident). These scripts will add or remove the interface as a kismet source which is needed for kismet not going crazy. We can verify if these rules are working properly by connecting the devices and looking if all of the properties have been set:
2.6 Start the sshd and avahi-daemon services
The avahi-daemon and sshd services will allow you to easily connect to whitestar via ethernet using only IPv6 link-local addressing. This will work in every case where a PC is simply connected to the whitestar with a Ethernet cable. Enable and start both services using the standard systemctl commands:
# systemctl enable ssh avahi-daemon Synchronizing state of ssh.service with SysV init with /lib/systemd/systemd-sysv-install... Executing /lib/systemd/systemd-sysv-install enable ssh Synchronizing state of avahi-daemon.service with SysV init with /lib/systemd/systemd-sysv-install... Executing /lib/systemd/systemd-sysv-install enable avahi-daemon # systemctl start ssh avahi-daemon
2.7 Copy ssh key
Use ssh-copy-id to copy your public SSH key to the authorized_keys of the user that will run the kismet services.
2.8 Install Python bindings for kismet
In order to run the whitestar watchdog script you need to install the kismetclient Python library using pip.
2.9 Install kismet and gpsd
Install the kismet and gpsd packages using "apt".
2.10 Edit kismet.conf server configuration file
Edit the kismet server configuration file, change a few of the options listed below. The other parameters are ok with their default values.
/etc/kismet/kismet.conf
# Prefix of where we log (as used in the logtemplate later) # logprefix=/some/path/to/logs logprefix=/var/log/kismet [...] # Do we have a GPS? gps=true # Do we use a locally serial attached GPS, or use a gpsd server, or # use a fixed virtual gps? # (Pick only one) gpstype=gpsd # Host:port that GPSD is running on. This can be localhost OR remote! gpshost=localhost:2947 [...] # How often (in seconds) do we write all our data files (0 to disable) writeinterval=10 [...] # File types to log, comma seperated. Built-in log file types: # alert Text file of alerts # gpsxml XML per-packet GPS log # nettxt Networks in text format # netxml Networks in XML format # pcapdump tcpdump/wireshark compatible pcap log file # string All strings seen (increases CPU load) #logtypes=pcapdump,gpsxml,netxml,nettxt,alert logtypes=alert,netxml,gpsxml [...] # Default log title logdefault=whitestar [...]
These settings have the following effect:
- store logs in /var/log which which is where the hard disk is mounted
- enable using a GPS and make it reconnect to gpsd if the connection is lost
- make the kismet server to flush all logs every 10 seconds, this prevents the log data from being lost when power is cut. This happens frequently in a car without battery backup for the whitestar.
- store alrts, networks and GPS position logs, do not store pcap dumps, these take up too much space
- set the log title to "whitestar", the title is used to build log filenames
2.11 Create a user to run the kismet
We need to create a nonprivileged user to run the kismet monitor. In my whitestar this user is called 'enki'.
2.12 Setup log storage on external hard disk
You should create two partitions (or better LVs) on the external HDD. One of those will be for the swap partition, the other we will mount as /var/log.
2.13 Make journal dir in /var/log
If the /var/log/journal directory exists and systemd-journald has the default configuration "Storage=auto" it will store the journal in /var/log/journal. This is what we want as the /var/log filesystem will be stored on the external HDD which will provide us with a nice way to troubleshoot or collect statistics from the system by just unplugging the drive and taking it with us.
2.14 Setup fs and swap
Setup the two LVs as swap and /var/log filesystem in /etc/fstab. Remove the original swap device on the flash drive, it's a massively bad idea to put swap on flash. Take care to include the "nofail" flag in the /var/log filesystem entry. This prevents systemd from creating a "RequiredBy" dependency for local-fs.target which in turn makes the system drop to emergency mode if the filesystem is not available (for example when the HDD is not connected).
/etc/fstab
UUID="<UUID1>" none swap sw,nofail 0 0 UUID="<UUID2>" /var/log auto defaults,relatime,nofail 0 0
The UUID1 and UUID2 fields are UUID identifiers (from the 'blkid' command) for the swap device and the /var/log filesystem we set up earlier.
2.15 Systemd units for custom services
I have created some custom systemd services for:
- kismet server
- kismet client acting as monitor and running on /dev/tty1
- the watchdog which checks the state of all other parts and reports the status on the LED monitor
- the archiver which moves old kismet logs from the main kismet directory to the archive directory. This allows for easier automatic extraction.
- the OBD-II data logger
/etc/systemd/system/kismet-server.service
[Unit] Description=Kismet server RequiresMountsFor=/var/log/kismet [Service] ExecStart=/usr/bin/kismet_server -s ExecStartPost=/usr/local/bin/kismet-add-all-sources Restart=on-abnormal [Install] WantedBy=multi-user.target
The kismet-server systemd service file launches the kismet server as soon as /var/log/kismet becoms available (the external HDD needs to be mounted for this). After it starts it launches a script which adds all wireless interfaces marked before using udev rules as sources to kismet. Furthermore kismet server is restarted when it crashes (which happens more often than you'd imagine).
/etc/systemd/system/kismet-monitor.service
[Unit] Description=Kismet monitor [Service] ExecStart=/usr/bin/kismet_client User=enki Group=enki Restart=always RestartSec=10 StandardOutput=tty TTYPath=/dev/tty1 [Install] WantedBy=basic.target
The kismet monitor runs bound to the /dev/tty1 terminal which means, that it should always be visible when you connect a monitor to the whitestar. It's started as the user we created before ('enki' in our case) and is always restarted when it breaks.
/etc/systemd/system/whitestar-watchdog.service
[Unit] Description=Whitestar watchdog [Service] ExecStart=/usr/local/bin/whitestar-watchdog.py Restart=always RestartSec=1 [Install] WantedBy=multi-user.target
The watchdog script monitors important aspects of whitestar operation and displays this information as status LEDs on the random board of LEDs attached to the whitestar PC via USB.
/etc/systemd/system/whitestar-archiver.service
[Unit] Description=Gathered data archiver RequiresMountsFor=/var/log/kismet /var/log/archive After=kismet-server.service [Service] Type=oneshot ExecStart=/usr/local/bin/archive-kismet-data.sh [Install] WantedBy=multi-user.target
The archiver script makes tarfiles out of all previously gathered kismet logs and moves them from /var/log/kismet to /var/log/archive. There they are available for pickup after connecting to whitestar using the 'archiver' account.
/etc/systemd/system/obdgpslogger.service
[Unit] Description=OBD GPS logger RequiresMountsFor=/var/log/obdgpslogger Requires=dev-obd.device [Service] ExecStart=/usr/bin/obdgpslogger -s /dev/obd -b 38400 -t -l /var/log/obdgpslogger/serial.log -d /var/log/obdgpslogger/db -i temp,rpm,vss,maf,throttle -u /var/log/obdgpslogger/out.log Restart=on-abnormal [Install] WantedBy=multi-user.target
The obdgpslogger is used to log OBD-II data. The baudrate of the OBD-II interface depends on the manufacturer and should be adjusted (try different baudrates using 'picocom' until you get one where you get an answer from the chip for the 'ATZ' command). The obdgpslogger waits for both the external HDD filesystem as well as for the ELM327 device to appear. I'm logging everything including the serial port log for debugging purposes.
2.16 Enable all services to start at boot
systemctl enable kismet-monitor.service kismet-server.service obdgpslogger.service whitestar-archiver.service whitestar-watchdog.service
2.17 Disable NetworkManager
As we won't be using NetworkManager but instead rely on systemd-networkd we can disable the NetworkManager service:
# systemctl disable NetworkManager
2.18 Systemd .network for LAN link
The Ethernet device on the whitestar PC will be used to make an SSH connection for debugging and downloading of the gathered wardriving data. For this we need for some service to bring the internal Ethernet device up when a cable is plugged. The simplest way to do this (as we already have systemd) is to create a simple "connection" description for systemd.
/etc/systemd/network/ether.network
[Match] MACAddress=e0:cb:4e:2e:2b:70 [Network] Description=Local Ethernet network
What this network file will do is bring up the internal Ethernet interface (specified using it's MAC address) using only IPv6 local addressing. This is convenient for us as it allows to use the connection even when we just plug in a cable straight from the whitestar to a PC without a DHCP server present.
2.19 Setup archive directory
Make a /var/log/archive directory so that the archiver can store the kismet logs for retrieval.
2.20 Setup archive file transfer account
Create a user called 'archive' with the home directory pointing to /var/log/archive. This user will be used to download the log files from the whitestar via SSH for further processing. Set up SSH public key authentication for this account. Restrict it to only have sftp accessible, the following sshd_config snippet should come in handy:
Match User archive
KbdInteractiveAuthentication no
PermitTTY no
ForceCommand internal-sftp
In order to get the data from whitestar and onto wigle.net I unplug the HDD and take it home. There, after plugging it into my workstation a number of automatic steps are performed in the background by systemd resulting in the relevant files being copied from the HDD and uploaded to wigle.net. All of the code needed to do this is available in https://github.com/mgrela/whitestar-host and https://github.com/mgrela/tools
First, we need a way to launch code when the external HDD is connected to the PC. This is accomplished using the following udev file in /etc/udev/rules.d:
$ cat /etc/udev/rules.d/99-whitestar-disk.rules
ACTION=="add", SUBSYSTEM=="block", ATTR{partition}=="<PARTNO>", ATTRS{vendor}=="<HDD VENDOR> ", ATTRS{model}=="<HDD MODEL> ", TAG+="systemd", ENV{MANAGER_USER_WANTS}="whitestar-downloader.service"
The PARTNO, HDD VENDOR AND HDD MODEL parameters are used to identify a particular block device on the external drive, you can easily check for the relevant attributes using the udevadm info -a command:
$ udevadm info -a --name=/dev/sdb2
[...]
looking at device '/devices/pci0000:00/0000:00:1a.7/usb1/1-2/1-2:1.0/host6/target6:0:0/6:0:0:0/block/sdb/sdb2':
ATTR{partition}=="2"
[...]
looking at parent device '/devices/pci0000:00/0000:00:1a.7/usb1/1-2/1-2:1.0/host6/target6:0:0/6:0:0:0':
ATTRS{model}=="<HDD MODEL> "
[...]
ATTRS{vendor}=="<HDD VENDOR> "
These attributes belong to different sysfs nodes (a partition and a SCSI device) but because the ATTRS{} keyword in udev rules looks for attributes up in the device hierarchy this works. Ok, so now when the external HDD is plugged in the "whitestar-downloader.service" unit will be started by the user's systemd instance. I'm not sure if this will be the instance of the user that is currently logged in or any user which has this service installed.
In order to allow the user to mount the external HDD in a location expected by the script (/mnt/whitestar) an fstab entry needs to be created:
$ cat /etc/fstab | grep whitestar /dev/disk/by-uuid/b0974a72-8dcd-4b82-9338-85bcdbd2701c /mnt/whitestar auto defaults,relatime,nofail,noauto,user 0 0
In order to prevent this device from being automatically mounted when plugged in the 'noauto' flag is defined.
The whitestar-downloader.service unit takes care of moving the files we want from the external HDD to a directory in our $HOME. The unit just runs a companion script:
$ cat ~/.config/systemd/user/whitestar-downloader.service [Unit] Description=Whitestar files downloader [Service] Type=oneshot ExecStart=/home/enki/bin/whitestar-dl.sh
$ cat ~/bin/whitestar-dl.sh
#!/bin/sh
readonly SRCDIR=/mnt/whitestar
readonly DESTDIR=$HOME/devices/whitestar
readonly DIRS="archive"
mount "$SRCDIR"
shopt -q nullglob
echo "Downloading files from '$SRCDIR' to '$DESTDIR'" >&2
pushd "$SRCDIR"
for path in $DIRS; do
tgtdir="$DESTDIR/$path"
mkdir -p "$tgtdir"; pushd "$tgtdir"; rsync --remove-source-files -av "$SRCDIR/$path/" "$DESTDIR/$path"; popd
done
popd
umount "$SRCDIR" || true
After this unit runs the files end up being copied to $HOME/devices/whitestar/archive. The progress can be tracked in the journal:
wrz 03 23:23:22 shoggoth slim[873]: "/org/freedesktop/UDisks2/block_devices/sdb" has new interfaces: ("org.freedesktop.UDisks2.Block", "org.freedesktop.UDisks2.PartitionTable")
wrz 03 23:23:22 shoggoth slim[873]: "/org/freedesktop/UDisks2/block_devices/sdb1" has new interfaces: ("org.freedesktop.UDisks2.Block", "org.freedesktop.UDisks2.Partition", "org.freedesktop.UDisks2.Swapspace")
wrz 03 23:23:22 shoggoth systemd[929]: Starting Whitestar files downloader...
wrz 03 23:23:22 shoggoth slim[873]: "/org/freedesktop/UDisks2/block_devices/sdb2" has new interfaces: ("org.freedesktop.UDisks2.Block", "org.freedesktop.UDisks2.Filesystem", "org.freedesktop.UDisks2.Partition")
wrz 03 23:23:22 shoggoth kernel: EXT4-fs (sdb2): mounted filesystem with ordered data mode. Opts: (null)
wrz 03 23:23:22 shoggoth whitestar-dl.sh[31584]: Downloading files from '/mnt/whitestar' to '/home/enki/devices/whitestar'
wrz 03 23:23:22 shoggoth whitestar-dl.sh[31584]: /mnt/whitestar ~
wrz 03 23:23:22 shoggoth whitestar-dl.sh[31584]: ~/devices/whitestar/archive /mnt/whitestar ~
wrz 03 23:23:22 shoggoth whitestar-dl.sh[31584]: sending incremental file list
wrz 03 23:23:23 shoggoth whitestar-dl.sh[31584]: rsync: opendir "/mnt/whitestar/archive/.ssh" failed: Permission denied (13)
wrz 03 23:23:23 shoggoth whitestar-dl.sh[31584]: ./
wrz 03 23:23:23 shoggoth whitestar-dl.sh[31584]: kismet-2016-07-17T18:22:30+0000.tar
wrz 03 23:23:23 shoggoth whitestar-dl.sh[31584]: sent 409,895 bytes received 47 bytes 273,294.67 bytes/sec
wrz 03 23:23:23 shoggoth whitestar-dl.sh[31584]: total size is 409,600 speedup is 1.00
wrz 03 23:23:23 shoggoth whitestar-dl.sh[31584]: rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1178) [sender=3.1.2]
wrz 03 23:23:23 shoggoth whitestar-dl.sh[31584]: /mnt/whitestar ~
wrz 03 23:23:23 shoggoth whitestar-dl.sh[31584]: ~
wrz 03 23:23:23 shoggoth systemd[929]: Started Whitestar files downloader.
The $HOME/devices/whitestar/archive directory is watched for new files by a systemd .path unit:
$ cat ~/.config/systemd/user/whitestar-archive.path [Unit] Description=Whitestar archive path watcher [Path] PathExistsGlob=/home/enki/devices/whitestar/archive/kismet-*.tar Unit=wigle-uploader.service [Install] WantedBy=paths.target
This unit starts the wigle.net uploader tool when new files are being placed in the archive directory on our workstation. The wigle-uploader.service is started which just runs a helper script:
$ cat ~/.config/systemd/user/wigle-uploader.service [Unit] Description=Wigle.net uploader [Service] Type=oneshot Environment="WIGLE_USERNAME=enkiusz" "WIGLE_PASSWORDFILE=%h/.config/wigle-uploader.password" ExecStart=/home/enki/bin/wigle-uploader.sh
$ cat ~/bin/wigle-uploader.sh
#!/bin/sh
readonly SRCDIR=$HOME/devices/whitestar/archive
shopt -q nullglob
if [ -z "$WIGLE_USERNAME" -o -z "$WIGLE_PASSWORDFILE" ]; then
echo "WIGLE_USERNAME and WIGLE_PASSWORD environment variables need to be set" >&2
exit 1
fi
echo "Uploading files inside '$SRCDIR' as '$WIGLE_USERNAME'" >&2
# Gzip uncompressed tars, wigle.net only processes .tar.gz files properly
for t in $SRCDIR/kismet-*.tar; do
gzip $t
done
systemd-inhibit --what=idle $HOME/bin/wiglenet-uploader.py -l "$WIGLE_USERNAME" -p "$WIGLE_PASSWORDFILE" --delete-imported $SRCDIR/kismet-*.tar.gz
The wiglenet-uploader.py script is available in my tools repo. The password is stored in a file outside of the usual configuaration files so that I can easily share them here without the fear of any leaks ;). This is very good practice which I recommend to anyone who is publishing his dotfiles or other "personal" config files as public repositories. What results in deploying all of this good stuff is that .tar files placed in the $HOME/devices/whitestar/archive directory get automatically uploaded to wigle.net. As before, the progress can be tracked using the journal. An example run is provided below:
wrz 03 23:23:23 shoggoth systemd[929]: Starting Wigle.net uploader...
wrz 03 23:23:23 shoggoth wigle-uploader.sh[31592]: Uploading files inside '/home/enki/devices/whitestar/archive' as 'enkiusz'
wrz 03 23:23:23 shoggoth wigle-uploader.sh[31592]: INFO:requests.packages.urllib3.connectionpool:Starting new HTTPS connection (1): wigle.net
wrz 03 23:23:24 shoggoth wigle-uploader.sh[31592]: INFO:root:Successfuly authenticated to 'https://wigle.net/' as identity 'enkiusz', got session id '680063891'
wrz 03 23:23:24 shoggoth wigle-uploader.sh[31592]: INFO:root:Uploading a batch of '1' source files
wrz 03 23:23:24 shoggoth wigle-uploader.sh[31592]: INFO:root:Uploading source file '/home/enki/devices/whitestar/archive/kismet-2016-07-17T18:22:30+0000.tar.gz' (19387 bytes)
wrz 03 23:23:25 shoggoth wigle-uploader.sh[31592]: INFO:root:Filename '1472937805_kismet-2016-07-17T18_22_30_00000' from source '/home/enki/devices/whitestar/archive/kismet-2016-07-17T18:22:30+0000.tar.gz' has transid '20160903-00637'
wrz 03 23:23:25 shoggoth wigle-uploader.sh[31592]: INFO:root:Filename '1472937805_kismet-2016-07-17T18_22_30_00001whitestar-20160717-20-19-55-1.netxml' from source '/home/enki/devices/whitestar/archive/kismet-2016-07-17T18:22:30+0000.tar.gz' has transid '20160903-00638'
wrz 03 23:23:25 shoggoth wigle-uploader.sh[31592]: INFO:root:Filename '1472937805_kismet-2016-07-17T18_22_30_00002whitestar-20160717-20-19-55-1.gpsxml' from source '/home/enki/devices/whitestar/archive/kismet-2016-07-17T18:22:30+0000.tar.gz' has transid '20160903-00639'
wrz 03 23:23:25 shoggoth wigle-uploader.sh[31592]: INFO:root:Filename '1472937805_kismet-2016-07-17T18_22_30_00003whitestar-20160717-20-19-55-1.alert' from source '/home/enki/devices/whitestar/archive/kismet-2016-07-17T18:22:30+0000.tar.gz' has transid '20160903-00640'
wrz 03 23:23:25 shoggoth wigle-uploader.sh[31592]: INFO:root:4 transactions from current batch still pending, total 0 transactions completed successfuly, total 0 transactions failed
wrz 03 23:23:35 shoggoth wigle-uploader.sh[31592]: INFO:root:4 transactions from current batch still pending, total 0 transactions completed successfuly, total 0 transactions failed
wrz 03 23:23:46 shoggoth wigle-uploader.sh[31592]: INFO:root:4 transactions from current batch still pending, total 0 transactions completed successfuly, total 0 transactions failed
wrz 03 23:23:56 shoggoth wigle-uploader.sh[31592]: INFO:root:4 transactions from current batch still pending, total 0 transactions completed successfuly, total 0 transactions failed
wrz 03 23:24:06 shoggoth wigle-uploader.sh[31592]: ERROR:root:Processing of 'kismet-2016-07-17T18_22_30_00000' has failed
wrz 03 23:24:06 shoggoth wigle-uploader.sh[31592]: INFO:root:3 transactions from current batch still pending, total 0 transactions completed successfuly, total 1 transactions failed
wrz 03 23:24:16 shoggoth wigle-uploader.sh[31592]: ERROR:root:Processing of 'T18_22_30_00003whitestar-20160717-20-19-55-1.alert' has failed
wrz 03 23:24:16 shoggoth wigle-uploader.sh[31592]: INFO:root:Processing of '/home/enki/devices/whitestar/archive/kismet-2016-07-17T18:22:30+0000.tar.gz' has been successful, 0 new WiFi APs w/ GPS (0 total WiFi APs w/ GPS)
wrz 03 23:24:16 shoggoth wigle-uploader.sh[31592]: INFO:root:Processing of '/home/enki/devices/whitestar/archive/kismet-2016-07-17T18:22:30+0000.tar.gz' has been successful, 0 new WiFi APs w/ GPS (0 total WiFi APs w/ GPS)
wrz 03 23:24:16 shoggoth wigle-uploader.sh[31592]: INFO:root:Deleting source file '/home/enki/devices/whitestar/archive/kismet-2016-07-17T18:22:30+0000.tar.gz' with '2' files successfuly imported ('2' failures)
wrz 03 23:24:16 shoggoth wigle-uploader.sh[31592]: INFO:root:0 transactions from current batch still pending, total 2 transactions completed successfuly, total 2 transactions failed
wrz 03 23:24:26 shoggoth systemd[929]: Started Wigle.net uploader.
As you can see there are 0 new APs discovered as this is not the first time I have performed this test ;).
3.22 Mothership uplink
Because we now live in a world of serious IoT the whitestar needed a uplink to the mothership in order to stay edgy :). This is now implemented to upload the gathered log files directly to a server. The solution has a few different parts:
- a Sierra Wireless 307 USB 3G modem connected via USB to the whitestar
- a PPP connection using a custom optimized chat script that gets activated every time the whitestar boots
- a OpenVPN client which connects to my router
- an rsync script which sends the content of the archive directory (/var/log/archive) to a remote server behind my VPN.
First, because we don't use ModemManager and any of that and the Sierra Wireless modem exposes 4 different ttyUSB ports we need to simplify things for pppd. We match the UART port that is the AT interface and mark it to be /dev/modem:
root@whitestar:~# cat /etc/udev/rules.d/30-sierra-wireless-307.rules
# Sierra Wireless 307 3G modem
SUBSYSTEM=="tty", SUBSYSTEMS=="usb", DRIVERS=="sierra", ATTRS{bNumEndpoints}=="03", SYMLINK+="modem"
root@whitestar:~#
This device is used by a pppd and chat script to establish a GPRS connection:
root@whitestar:~# cat /etc/chatscripts/aero2 ABORT 'BUSY' ABORT 'NO CARRIER' ABORT 'ERROR' ABORT '+CME ERROR: 100' '' ATZ OK ATH OK ATE1 OK AT+CGDCONT=1,"IP","darmowy" OK ATD*99***1# CONNECT
root@whitestar:~# cat /etc/ppp/peers/aero2 /dev/modem 115200 debug maxfail 0 persist noauth noipdefault defaultroute usepeerdns user darmowy password darmowy connect '/usr/sbin/chat -v -t 50 -f /etc/chatscripts/aero2'
There is a systemd service unit overseeing the pppd process:
root@whitestar:~# cat /etc/systemd/system/ppp@.service [Unit] Description=PPP connection to %I Requires=dev-modem.device [Service] ExecStart=/usr/sbin/pppd nodetach call %I [Install] WantedBy=multi-user.target
This gives us an almost always on PPP connection:
root@whitestar:~# ip a show dev ppp0
3: ppp0: <POINTOPOINT,MULTICAST,NOARP,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UNKNOWN group default qlen 3
link/ppp
inet x.x.x.x peer 10.64.64.64/32 scope global ppp0
valid_lft forever preferred_lft forever
On top of this an openvpn client configuration file is used
root@whitestar:~# cat /etc/openvpn/kraken-client.conf verb 3 client remote kraken.strangled.net 1194 proto tcp remote-cert-tls server float comp-lzo #redirect-gateway ping-restart 30 <ca> [...] </ca> <cert> [...] </cert> key kraken-client.key <dh> [...] </dh> dev tun.bukavpn
Also supervised by a systemd unit file. This time it's the standard openvpn@.service shipped with debian. This in turn gives us an almost always-on TUN connection to the VPN that gives us access to a backend server for collecting the wardriving logs.
# ip a show dev tun.bukavpn
6: tun.bukavpn: <POINTOPOINT,MULTICAST,NOARP,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UNKNOWN group default qlen 100
link/none
inet 10.100.0.3/24 brd 10.100.0.255 scope global tun.bukavpn
valid_lft forever preferred_lft forever
All of this is used to run an rsync which uploads contents of /var/log/archive to a remote server. This is acomplished by a simple script that simply attempts the rsync command every 2 minutes. It doesn't care about failures, it just retries on failure. Also, the files that were successfuly transferred are removed from the source dir which is useful.
# cat /usr/local/bin/whitestar-uploader.sh #!/bin/sh while true; do rsync --remove-source-files -av /var/log/archive whitestar@172.20.171.116:log sleep 120 done
This script is started by a systemd unit taking care of running it only after the archiver script has ended collecting the completed capture files from /var/log/kismet.
root@whitestar:~# cat /etc/systemd/system/whitestar-uploader.service [Unit] Description=Whitestar watchdog After=whitestar-archiver.service [Service] ExecStart=/usr/local/bin/whitestar-uploader.sh Restart=always RestartSec=1 [Install] WantedBy=multi-user.target root@whitestar:~#
I know that the whole VPN bit seems a bit overengineered, but meh. You can simplify it yourself if you want. Also, now the files land on the mothership server and nothing uploads them anywhere, something that is next on my work list :).
2.22 Final time measurements
Time to Capture == 28 sec from poweron
19 seconds bios,
# systemd-analyze Startup finished in 9.242s (kernel) + 13.000s (userspace) = 22.242s
# systemd-analyze blame
4.229s systemd-networkd-resolvconf-update.service
2.602s kismet_archiver.service
2.386s dev-sda1.device
2.238s systemd-journal-flush.service
2.098s systemd-networkd.service
1.225s networking.service
1.111s gpsdctl@ttyUSB0.service
878ms systemd-logind.service
592ms systemd-journald.service
573ms systemd-udev-trigger.service
571ms kismet_server.service
569ms console-setup.service
547ms grub-common.service
531ms apport.service
521ms ondemand.service
469ms systemd-user-sessions.service
445ms user@1000.service
440ms pppd-dns.service
423ms lm-sensors.service
344ms var-log.mount
342ms systemd-modules-load.service
337ms systemd-tmpfiles-setup-dev.service
281ms avahi-daemon.service
202ms systemd-udevd.service
165ms speech-dispatcher.service
150ms systemd-sysctl.service
124ms alsa-restore.service
103ms systemd-remount-fs.service
96ms dev-mqueue.mount
95ms kmod-static-nodes.service
92ms systemd-update-utmp.service
82ms plymouth-quit-wait.service
76ms systemd-update-utmp-runlevel.service
74ms plymouth-quit.service
73ms ufw.service
70ms plymouth-read-write.service
62ms dev-disk-by\x2duuid-9152004c\x2d0317\x2d45c0\x2d9482\x2d1baa6c4e81fa.swap
60ms systemd-vconsole-setup.service
58ms sys-kernel-debug.mount
56ms sys-fs-fuse-connections.mount
53ms systemd-timesyncd.service
49ms systemd-rfkill@rfkill2.service
46ms systemd-tmpfiles-setup.service
45ms rc-local.service
37ms dns-clean.service
36ms ureadahead-stop.service
28ms resolvconf.service
26ms systemd-random-seed.service
22ms systemd-rfkill@rfkill0.service
22ms systemd-rfkill@rfkill1.service
16ms systemd-backlight@backlight:acpi_video0.service
# systemd-analyze critical-chain
The time after the unit is active or started is printed after the "@" character.
The time the unit takes to start is printed after the "+" character.
multi-user.target @9.919s
└─rc-local.service @8.830s +45ms
└─network.target @8.815s
└─systemd-networkd.service @6.715s +2.098s
└─dbus.service @6.459s
└─basic.target @6.160s
└─sockets.target @6.160s
└─acpid.socket @6.160s
└─sysinit.target @6.158s
└─systemd-rfkill@rfkill2.service @9.217s +49ms
└─system-systemd\x2drfkill.slice @5.452s
└─system.slice @484ms
└─-.slice @482ms
2.22 Future plans
When six out of the below goals will be achieved I will consider the whitestar to be proper Vorlon technology :).
2.22.1 Other upload destinations
- Mozilla stumbler (OK, this one is not opensource but still better than Google ;)
- openbmap.org
2.22.3 Better OBD-II integration
While the whitestar already logs OBD-II data using obdgpslogger, a proper integration would allow for better fuel economy tracking than the paper-and-pencil method I'm using now.
2.22.4 Improved statistics & monitoring
I feel the need to better monitor the wardriving performance and environmental factors. Measurements like amount of packets captured for each 802.11 card vs. current cruise speed would allow to properly determine the best strategy for capturing most information.
2.22.5 Cell information capture
A different form of radio beacon that is useful for location services are cellphone tower IDs. There are a number of ways for those to be captured, the simplest one would be to connect a cellphone modem to the PC periodically fethcing the current visible cell tower IDs from it. I'm also considering other approaches such as using multiple osmocom-bb compatible phones for parallel GSM spectrum scanning. This should achieve better results as you won't be limited to the cell towers for your home network.
2.22.6 GPS receiver sharing with cell phone
The GPS receiver that I have connected to the system achieves higher TTFF than my cellphone. It would be thus good to be able to feed the GPS sentences from the phone to kektop until the whitestar's GPS receiver gets a fix.
2.22.7 GPS used to provide time
GPS should be usable as a time synchronization source for ntpd running on the whitestar.
2.22.8 Zero-flash-writes
The goal of this setup is to achieve zero writes on the internal flash to increase it's endurance. For this to be achieved a system of monitoring and logging processes which perform frequent writes to the flash needs to be built.
2.22.9 Navigation integration
It would be nice if the phone navigation could be instructed to guide the user through a particular route that is optimal for wardriving. For example, routes could be taken which take us onto roads or parts of the city where the amount of known wifi networks is scarce.
2.22.10 Fully automatic and practical data upload
The mechanism to extract information gathered by the device has to be improved. Currently, I just unpug the disk and connect it to my workstation. A networked solution would be preferable. I do not want to use the wireless cards to connect to an external device via WiFi to not interfere with scanning, therefore a cable based solution would be best.
2.22.11 Debug marking
There are certain situations when driving with the whitestar working where the operator notices that something doesn't look right. This is usually because the LEDs on the monitor box indicate that there is some kind of problem. In this case (as everything should be logged to the journal anyway) it would be good to be able to press a button on the monitor box (the button can be seen on the photos) and "mark" a point in time in the logs so that it can be easily found later and the problem investigated.