Dan Maloney
Hi all, welcome to the Hack Chat. I'm Dan, I'll be moderating today for our guest Michelle Thompson, who's here to tell us about what goes into designing hardware challenges for cons. I'm not sure I saw her log in yet, though - are you out there, Michelle?
Yes! Hello and thank you so much.
Hi Michelle, welcome to the Hack Chat!
I very much appreciate the opportunity. I have some things written to start us off.
Greetings all! I'm Abraxas3d (Michelle Thompson W5NYV), MSEE from USC in Information Theory, Senior Member IEEE, IEEE Vice Chair San Diego Chapter Information Theory Society, IEEE Distinguished Visitor 2017-2019, co-founder and current CEO of Open Research Institute, Inc., Chair of GNU Radio Conference, ARRL Life Member, 10-10 Life Member, Yikes Did I Let my TAPR Membership Lapse Again?, AMSAT Director 2019-2021, DEFCON, Burning Man, Pokemon, and Formula 1! Thank you for welcoming me here today.
I've coordinated a Capture the Flag (CTF) competition at GNU Radio Conference over the past few years. CTFs are competitions that are very much like sports events. There are challenges that require skill to solve. Points are awarded. There is usually a limited time, sometimes a day or a weekend. There's often prizes, sometimes cash awards. They go on all the time - there is usually one every weekend that you can do remotely.
Most CTFs are software-based and information security focused. They are somewhat easy to do remotely.
For GNU Radio Conference, we wanted to do something that let people use the DSP framework with their software defined radios to figure out puzzles from real hardware on site at the event. This meant real hardware (and software) for the challenges and real hardware and software used by the participants.
So what kind of puzzles or challenges have we had?
Many modern signals sound exactly like noise in a general receiver, so sometimes just figuring out they are even there is a big first step. There are several types of radio signals that can lurk beneath the noise floor, like spread spectrum. Some hop around, like you might have seen near 915MHz in the US. Not knowing where the next burst of information will show up can make intercepting a signal tricky!
In order to receive, you have to demodulate a signal. That means figure out what shape or kind of electromagnetic waveforms are being used, how large is the set of waveform symbols (sort of like what alphabet a human language uses), and how to best capture those wiggly waveforms.
Once that's known, for a digital signal, you then should have a lot of 1's and 0's. If it's encoded, to protect from errors or to provide secrecy, then you have another layer to work through. And so on! There may be multiple levels and different types of coding. There are some codes that are theoretically impossible to crack. There are some that are possible to crack but take too long to realistically spend time on. For friendly competitions, we use modulation and coding that aren't too difficult. There may be a clever trick, or a technique with special utility. Our emphasis has been on friendly competitive learning, and hints and explanations are OK during the competition.
A great walk-through of reverse engineering a signal is Matt Knight's GNU Radio Conference presentation on LoRa:
You can see how you peel back the onion layers!
There are many modern radio signals that use protocols that mix data types, like in Radio Data System, an FM broadcast format. https://en.wikipedia.org/wiki/Radio_Data_System
What we did with an RDS challenge is set up a very low power (legal) FM station at the conference. We created some content, programs, commercials, and so on. Some of answers could be found in the main audio content. However, RDS has several subcarriers. And we used them as well, to communicate information like the weather report, but we altered them slightly, so that the subcarrier disappeared if you used the standard GNU Radio RDS receiver. If you changed the subcarrier frequency, the data could be received again. You would notice this if you looked at a spectral display of the signal, and saw the subcarrier appearing in an unexpected place. Knowing where the subcarriers should be could be found from the published standard.
Taking something familiar and changing a parameter is a common design pattern for signals based challenges.
Recently, Clayton Smith VE3IRR, a regular at GNU Radio Conference, designed a very neat challenge for BSides Ottawa CTF. He used an OFDM signal to paint a visual picture on the the spectrum, so that you could see the image in the waterfall display. This is a time-honored technique in hardware signals CTFs and is a lot of fun to use. The GNU Radio module is called gr-paint and is by Ron Economos.
However, the image in this challenge was not just transmitted flat. It was compressed to the point where the relevant part was so squished you could not easily see it! Here is his blog post about it: https://irrational.net/2019/11/30/trolling-ctf-players-with-gr-paint/
Another series of challenges at GNU Radio Conference had to do with BlueTooth, which is almost endless in possibilities because of how many different ways the standard can be... "interpreted".
There is an entire self-contained BlueTooth CTF that is very well done here:
https://github.com/hackgnar/ble_ctf
All you need to deploy this is an ESP32. The scoring is done by connecting to and interacting with an ESP32 programmed with this codebase.
We had a challenge where you had to reverse engineer restaurant pager system. A common first step in reverse engineering is to look up the FCC ID which is usually somewhere on the device. This lets you find out all sorts of great information about the thing you're working with. However, this pager system didn't have an FCC ID.
We asked the participants to "clear the room" - make all the pagers go off at once, among other things.
We had a challenge where you had to intercept the photo transmitting protocol of Yaesu System Fusion ham radio HT, and identify something in the JPG.
We had a challenge where we put a remote control car under a turned-over bin with bluetooth controlled lighting. There was a color NTSC camera in the bin. If you could figure out how to control the remote control car (capturing, reversing, and replaying the signal from the remote control), and if you could figure out how to turn on the LED lighting in the box, and if you noticed the NTSC backup camera signal, and if you could figure out how to receive color NTSC, then you could get a series of flags for this challenge.
There's a writeup here: https://github.com/Abraxas3d/JerichoReport-
For this year's GNU Radio Conference event, we had planned an auto racing themed CTF with lots of automotive hardware on site to manipulate, signals to intercept, and even a retired race car that we were hoping we could let people hack on. Planning was well underway!
And then... we went virtual. There was no way I could see how to do this sort of event online. We postponed the CTF until 2021 and didn't think any more about it. I was disappointed, but the rest of the show needed a lot of work to convert to virtual and there was always next year.
And then I got the chance to compete in the Hack-a-Sat competition. This was sponsored by the Air Force and was a qualification event for the finals at DEFCON, recently concluded.
Instead of cancelling the event or postponing it - they moved the event, which was very hardware-focused and very hands-on sounding, online!
I was very curious to see how this would work!
The qualifications turned out great. We relied heavily on GNU Radio to take recorded IQ files and figure out the signals. In other challenges, we had to calculate orbits or deal with map data. I realized that having the hardware on site wasn't necessary to put on a respectable event.
I was invited to compete in the Hack-a-Sat finals, and expected it to be the same or similar to the Qualifications. It was not! Each finalist team got a flat-sat, or terrestrial engineering version of a satellite. We used this satellite to figure out how to regain control of a similar satellite "in orbit" that had been taken over by Bad People. Instead of intercepting and analyzing signals, this competition was largely about how to wrest network control back using Core Flight Services and COSMOS. So, more like a traditional CTF, with the big exception that we had hardware to work with. Well, some of us did - the team was distributed all over the United States, so only one household could put their hands on the hardware. This is a drawback of virtual events, but we all worked together to figure things out and verify them on our own flat sat before trying it on the official CTF server. The server "stood in" for the target satellite.
So, after this, we were inspired. Instead of converting the auto racing CTF to virtual, we started over and developed a linear series of challenges that relied heavily on pre-recorded radio data. You download these files and open them in GNU Radio. The data is presented to the flow graph as if it came off the air. While this does limit things, and makes the event less flexible, it does work. We were back in business and put together a CTF with characters, a plot, and a theme. You solve the mystery of a stranded hiker who turns out to be more than he seems.
You don't even need hardware to have a great puzzle event. Zhaolong Li in June IEEE Microwave Magazine wrote about "Gaming in Microwave Engineering Education." What he does with his students is that they all take a real data sheet for a real part, and (potentially) change something. The students then "sell" the part or board or filter or whatever, trying to convince the rest of the class that it's totally legit. These are called the "exaggerators". The audience is full of "challengers" that try and guess the parameter changed.
Now, the data sheet might be unchanged - totally legit! Or something might have been changed. The goal is to try and figure out what parameter no longer makes sense. The audience has two chances. If they guess the changed parameter, they win. If they fail, then the exaggerator wins.
For example, in one of the examples in the article, the gain of a horn antenna was altered.
So, for virtual events, relying much more on pre-recorded signals, while expanding the type of challenges to include more traditional information theory content, was how we were able to get back on track with hardware challenges at GNU Radio Conference. The conference will be 14-18 September (coming right up!) and you can find out more about it and attend (for free) at https://gnuradio.org
The CTF will run all week and there is a dedicated Matrix chatroom for it. All the GNU Radio flow graphs and Python code and MATLAB scripts will be released in a GitHub repository after the CTF is over.
Hello everyone!
hi
whoa, that's a whole blog post
:)
So, your interest in radio seems to have started early. What kicked it off?
I grew up in a household of amateur radio operators, so using radios and building circuits to do radio frequency things was normal.
We had some early computers too - Iearned how to code by taking over the VIC-20 that my dad bought because he was getting pressure to "do something" about computers at work.
Combining them with packet radio was ordinary, and packet networks in Arkansas at the time (mid 1980s) were up and running.
It just wasn't very unusual to use radios, or hear repeater traffic, or see HF operation at home. Outside the house this was a different story. Very few of my friends grew up with this sort of technology exposure.
If there was technology, then it was for consuming entertainment.
It still is.
I think this is still the same way today, where there is a huge amount of technology in a cellular phone, but it is usually used as a platform for delivering entertainment.
Yes concur with Land.
Oops Lane*
Regarding the GNU Radio Con CTF - how much advance notice did you have that the challenge would have to be retooled due to COVID-19?
I seem to recall the driving force behind WWW expansion was porn videos.
That is a good question. We moved somewhat early on compared to some other conferences, to move to virtual.
The more you know///
But, we did not really appreciate the fact that we could put on the CTF until after the Hack-a-Sat qualification round.
Also, how does ham packet radio work to provide (presumably non-commercial) WWW access to remote places? Is this a thing that's actually happening, or a fever dream only I have?
looks like that was late May
June is when we first started putting together challenges and retooling.
packet is not a replacement for the web, but can, has, and does extend data communications into all sorts of places. Here, where I live, in San Diego, there is a history of cross-border data networks really helping underserved communities in the past. Since amateur radio must be non-commercial, there is a hard limit on how it can be deployed.
A platform called "WebEOC" had a plug-in for packet radio that was actually used.
Winlink is another notable thing in this area.
It has made a lot of inroads in maritime use.
What I wanted to know, thanks Abraxas3d and G Mallery.
With events like GNU Radio Conference, that have been and are run by volunteers, the sort of adjustments that have been necessary to put on large events can be incredibly disruptive and stressful. I'm sure everyone that has volunteered for something like the Superconference knows this!
These challenges look really impressive and fun. Are you aware of some good hardware/software challenges catered towards middle/high school age students with very little to no start-up costs? Looking for some fun activities to do with my students.
Making adjustments to remote or virtual work, especially with competitions that were designed to work well in person with equipment you could or had to touch, meant some compromises.
Yes - there are some for middle school and high school. There are some WiFi based challenges, mostly from DEFCON WiFi Village, that are really quite good for middle-high school.
You don't need advanced equipment, WiFi is familiar, and it's cheap to deploy.
They can get really hard, so if you have a crew of students that dig in, you don't run out of content.
This is very relevant, because of something about competitions like this - the way that they are designed, can be very accessible, or very not-accessible. If you have lots of easy on-ramps, where the challenges are easy and engaging, then you increase confidence fast.
So, for education, like the data sheet competition from the IEEE article, you get a lot of traction.
The BlueTooth ESP32 might be good for high schoolers. The only drawback to that set of code and equipment is that the flag formatting is very finicky, which can cause frustration.
The way that many CTFs format the flags are like this:
Flag{answer-is-here}
And it's case sensitive. So hex strings or something long or formatted just slightly wrong will not "take"
There is a balance between autonomous scoring, which is great from the point of view of the organizer, and easy flag submission, which is what any participant would want.
Thank you so much for your response! I was looking through the IEEE article. I like that idea! The problem with challenges like the BLE CTF is that our students have very locked down chromebooks without access to the rudimentary chromebook shell. It is challenging to find activities to do in this environment.
One of the things we have found with remote SDR competition is that it's really hard to have an interactive or transmit-based challenge. But, there is at least one way around that.
The point about the Chromebooks is a good one. I have some Chromebooks here. Let me think about this and see if I can't find something that can be done with them. Anything HTML based or server based should work, which opens up a lot of traditional information security CTFs. But for hardware, maybe there is something overlooked.
One way to make transmitting work is to use GNU Radio to record the required transmitted packet or signal. If it "meets spec" or performs with respect to whatever protocol was required, then it can pass a test and capture the flag. Interactive can be done for low bandwidth signals and a server, but it has to be pretty narrow.
Thanks! Do you have any links to entry-level wifi-based challenges like the DEFCON wifi village you discussed? These may be suitable if I can get some loaner Raspberry Pis.
Yes - let me find either a starting point or get you a contact in WiFi Village.
Here's a page for getting started and setting up - this assumes you have at least a Raspberry Pi, and preferably something like a HackRF or other SDR:
http://sdr.ninja/training-events/sdr-wctf/
The best starting point for GNU Radio, where you start to assert more control over your radio and tell it exactly what to look for, is the set of tutorials here:
https://wiki.gnuradio.org/index.php/Tutorials
Would HF work for the source end of a CTF challenge? Maybe leveraging WSPR or some other beacon stations that participants can receive with SDRs?
Yes, it would - and it would be adventurous and fun.
Awesome thanks so much! Maybe next year when we are in-person and able to share materials, I'll purchase a class SDR to play around with.
We have used local VHF repeaters in the past as part of the CTF, where a net control was involved and in on it.
HF is definitely usable.
Yeah, just thinking HF since it has a potentially worldwide footprint
We talked about incorporating HF beacons, the Shortwave Radio Net (https://swradiogram.net/), and several other really neat HF centric things.
With so many remotely accessible ham stations, people wouldn't even need their own radio equipment.
For this year, we haven't set anything up formally. Mainly because propagation is kind of not great right now, and we didn't want participants to get too frustrated with that.
Yes - exactly right. Remote stations and web SDR really alleviates a lot of burden on setting up an HF station.
Drat you, Sun!
The majority of the challenges we've done have been higher frequency than HF, for several reasons. The antenna sizes are reasonable and the data rates are larger. But, HF is not forgotten. Given all the contests going on all the time, a good entry level CTF challenge might be a verified log entry from your station in some particular contest.
There is a proposal for an advanced contesting league for HF contests - one that incorporates a lot of the things we find super fun (digital modes, puzzles). It's called Blue Moon League and would operate on any fifth Saturday of the month.
This comes from the Ham 2.0 group (Ward Silver et al).
The main impediment for using HF so far, from any of the events I've done firsthand, is propagation.
Dan is right, drat you sun.
Would probably want to make it clear whether or not entries from users unlicensed in their country of origin will be accepted/rejected.
That's a really cool idea! Love anything to make ham less about talking and more about tech
There are a lot of very interesting things on HF. Radar, ionospheric sounding, jamming. Any of those are legit targets for a CTF signals competition.
*Looks at HF transceiver on desk* Sigh...
Yes this is a good point. While most of amateur spectrum is coordinated worldwide, some of it is not. Part of having the license is knowing what you can and cannot do with it, but any contest should be as open as possible. Receiving is in general completely free. Transmitting is not, so anything that requires transmitting needs to be as open as possible.
One of the things that GNU Radio Conference has done on occasion is provide hardware. Last year we gave the first 400 or so an RTL-SDR. We have had the great honor of being able to host Analog Devices workshops where PLUTOs are given away to every workshop attendee. When designing a hardware CTF, you have to think through what a typical attendee might have access to (in person).
For a virtual event, this concern is somewhat reduced, but not eliminated. We have to take care not to design challenges that require equipment that is uncommon or really expensive.
Now, this is the good news. We really do live in a golden age, where software defined radios are plentiful, powerful, and relatively cheap.
But, they all present a remarkably steep learning curve.
The hardware isn't the problem. The software definitely is.
You could always separate challenges in to different categories based on type. i.e. receive only tech for RDL dongles, midrange SDRs with transmit, etc.
Yes, that is an excellent point.
For 2017, we decided to make everything accessible with an RTL-SDR, and brought 25 of them as loaners. It worked - but we did run into limitations! It was good to do this, because then it was more fair.
At one of the recent DEFCONs, there was a hardware hacking village (hosted?) challenge, where you had to pick the locks on a box.
Very quickly, a big line formed and the throughput was slow.
That can happen when you set up something that requires particular hardware. Classic limited resource problem.
With signals based challenges, there isn't a line, really, in receiving the signal. Everyone can work on it in parallel. This is duplicated, somewhat, with distributing the radio wave files as IQ recordings. It's as if you received it off the air.
This is, arguably, not as satisfying as setting up your SDR, searching for the signal, finding it, refining your receiver, and recording it - but it is good enough to be able to have these sorts of events and get as much as possible out of them.
We've done a hardware challenge event in our hackerspace twice, I've done a writeup of the last time we did it - https://hackaday.io/project/18031. Thank you for talking about this! Let me see if I have some questions =)
OutstandingI I can't wait to read about it, which I will do ASAP.
Most of what we've done does not involve literal reverse engineering or modification of hardware (sometimes destructively) but there have been events where that is the method.
question - do you know of any repositories containing challenges for CTFs? Especially those that have decent hardware-related challenges. Wouldn't mind assembling gadgets for people to be hacked, even - that's the plan for some of the challenges I'll be doing myself.
There has been a reverse engineering challenge at DEFCON at the hardware hacking village, where you get a PCB and you have to do a set of things and answer questions about the circuit, the circuit function, or get it to do something it was definitely not intended to do starting out.
also, any writeups? Thank you for your initial post, it could probably stand on its own as a part of a great writeup or something =)
Check out ctftime.org for a LOT of CTF writeups
FYI, I'll post the transcript for the Chat right after we're done, for reference
You're very welcome. We will publish the current one after GRCon, I mention the BlueTooth ESP32 one, and Hack-a-Sat open sourced all of their challenges as well.
There are a lot out there!
Yes, thank you @Paul Williamson
Have you ever created or participated in a CTF that was too hard for anyone to crack?
I have participated in CTFs where challenges went unsolved, yes!
Don't forget to point conference attendees to https://hackaday.com/?s=radio for lots of helpful information :)
So far, we have not created any challenges that were not eventually solved, but the GNU Radio crowd is formidable.
I have tried to solve things, in some CTFs, where I didn't honestly even know where to start.
Then there are the challenges that were supposed to be hard, but also had an easy solution that the designer didn't think of.
It just required too much advanced knowledge or expertise, or was intended for a different audience.
And https://github.com/jopohl/urh
I felt a bit of that in the Hack-a-Sat finals, where my signals and receiver knowledge were literally not used. I had studied up on Core Flight Systems, but the challenges assumed a very high level, and I had basic competence.
Oh good point about easy solutions designers didn't think of. One year, I had a challenge where the answer was a color. Well, there are only so many colors... So, smart solvers simply ran through all the colors names and guessed the answer, once it was obvious that a color was involved.
All of the usual basic game theory and gamesmanship applies in CTF design!
Because of the enormous potential interdisciplinary potential of some of these competitions, this is why CTF teams are formed and win.
The top teams are very large, and put together strike forces to go do particular CTFs. They may have worked together for a long time, or they may recruit subject matter experts for a particular competition.
Looks like our time is up, so we'll have to let Michelle get back to getting ready for the con. Just want to say a big thanks to everyone for stopping by today, especially Michelle for giving us so much time in what must be a crunch for her. We really appreciate it!
For instance, I've designed a simple PCB with two ATTinys communicating between each other, one sending data to another, and that data is shown on a LCD display - the user needs to spoof that data and show their own info on the display. The hardware's ready, the idea is that the platform itself can make for multiple levels of complexity - depending on the firmware in both ATTinys. We also have a simple ATTiny-powered board that blinks an LED and you need to make it flash the LED in a different pattern. But also, whenever we do this challenge again, we'll have things like commercial fingerprint sensors, for example. We're also thinking about making a basic escape room.
Excellent chat, Michelle. You rock! Thank you.
thank you for sharing your knowledge! Will have to look into what you've linked when I'll be preparing for the next event =)
Thank you so much. It's so wonderful to be able to do this. Sophi and Dan are amazing people, and I really enjoy this community.
Yes thanks, I hope you get some great virtual participation.
Me too! Will be part of the write up.
hardware challenges after COVID... that might be tricky for whenever it will be that we'll be organizing our next event =)
Thanks all! Next week we'll have Tim "mithro" Ansell on to talk about the Skywater PDK
Discussions
Become a Hackaday.io Member
Create an account to leave a comment. Already have an account? Log In.