delx.au / blog

Guest post by lemnisca.

I’ve recently started getting into Linux wireless programming for work. I am a fairly experienced C programmer but don’t know much about kernel development so this is a new adventure for me. What I found was that while there is a lot of information available about how it all works — indeed, since it’s open source, one can at least in theory have complete information about the functioning of the system — there is not much in the way of an introduction. There’s no broad explanation of how it all works for someone completely new to the system. The documentation that does exist is largely focused on documenting individual functions, structs, etc. and is more of a reference for those already familiar with the system. I quickly found myself lost in a sea of source code, chasing down chains of function calls with no real feel for how the whole thing is put together. So I’ve decided to share what I’ve learnt, in the hopes that it might help someone else in the future.

Overview

Let’s start with an overview of the Linux wireless subsystem, from user space utilities all the way down to device drivers.

User space: ifconfig, iwconfig, iw, hostapd

The way that a regular user would interact with the Linux wireless stack is likely to be via high-level utilities like networkmanager: click the little WiFi logo in the system tray, pick a network and get online. Since we’re interested in getting down to how wireless drivers work, though, I’m going to skip that part and go straight to the command-line utilities. GUI configuration apps will typically just be a frontend for these anyway.

A wireless device can be configured using ifconfig just like any other network interface. However, this will only get you so far. To configure wireless-specific parameters, you will need to use other tools like iwconfig, iw and/or hostapd. You can use these to manually configure your wireless devices to, for example, join a specific channel or BSS, or act as an AP. These are the userspace utilities that are communicating directly with the kernel (via a netlink socket — more on that later). If you want to write some low-level wireless applications without actually venturing into kernelspace, checking out the source code of these programs would be a good place to start.

Inside the kernel: cfg80211, mac80211 and drivers

The first thing to understand about wireless drivers is that there are different types and the kernel thus has different ways of handling the different types of drivers. A wireless card can implement all or almost all of the 802.11 protocol in hardware or firmware (though generally excluding management stuff like associating with an AP), in which case it doesn’t need much help from the kernel — just a way to get the correct configuration parameters. On the other hand, a card can implement very little in hardware/firmware (so-called soft MAC cards), relying on the kernel to do almost all the work.

cfg80211 is the subsystem responsible for dealing with hard MAC drivers. It allows a driver to register an interface, along with a set of callbacks to be used to receive configuration parameters and events. Generally these will originally come from iw and friends. So for a hard MAC, what is happening when you click that little WiFi icon and select a network is that your GUI app is running one or more iw commands, and iw is in turn communicating the new configuration to cfg80211. cfg80211 then calls the appropriate callbacks in the driver, which handles passing the information on to the actual wireless device. This means that all wireless drivers have the same interface and are all registered with cfg80211 (at least for newer kernels/drivers).

For cards that only partially implement 802.11 in hardware or firmware, we have mac80211. mac80211 is a full implementation of 802.11 in software and is quite modular, so drivers are free to use all of it if they want, or only some of it if some of the functions are implemented in hardware for that card. A driver that uses mac80211 will register which operations it wants to handle and mac80211 will take care of the rest. mac80211 will then in turn be connected to cfg80211, so we still have the same interface for communicating with userspace.

Then of course we have the drivers themselves. Drivers are typically implemented as kernel modules and are responsible for registering the wireless interface with either cfg80211 or mac80211 and communicating with the hardware. So a driver will essentially consist of kernel module code, a set of callbacks for cfg80211 or mac80211, set up code to register the device and tell cfg80211/mac80211 which operations it has, and code for taking information from cfg80211/mac80211 and sending it on to the hardware and vice versa. They can also have some amount of processing in software, such as filtering out frames with CRC errors, handling frame aggregation, etc.

Connecting the dots: How do the different parts communicate?

Now we have an idea of what the different parts of the wireless stack are, we will look at how they are joined together and the interfaces between the different layers.

Userspace to kernelspace: netlink sockets and nl80211

Userspace utilities like iw use a netlink socket and the nl80211 netlink protocol to communicate with cfg80211. I’m going to write a more detailed post about this part soon, so keep an eye out for that for all the gory details. But in short: a userspace program can use libnl, which provides a nicer API than just using a raw netlink socket and does a lot of your work for you. The nl80211 protocol allows the userspace program to send commands and configuration information down into the kernel (such as setting the channel, or putting the card into ad-hoc mode), and receive information in return (such as scan results or the commands supported by the card).

cfg80211 and mac80211 communicate with drivers through a set of callbacks that the driver implements and registers when it starts up. Some of the callbacks the driver is required to implement but some are optional. This might be because they represent configuration options that are not supported by all cards, or perhaps they are functions that mac80211 has an implementation for, so it’s optional if the driver wants to provide its own. For cfg80211 these callbacks are contained in struct cfg80211_ops and for mac80211 they are in struct ieee80211_ops. In general the mac80211 versions of the callbacks will be prefixed by ieee80211_ and a driver’s implementation is prefixed by the driver name. For instance, ieee80211_tx is the mac80211 function to transmit a frame, and ath10k_tx is the equivalent function in the ath10k driver.

When a driver starts, it first needs to register. Let’s look at the process for a driver using cfg80211. The driver needs to create a wiphy, which contains some information about the interface such as its MAC address, RTS threshold, firmware version, etc. This can be done by calling the wiphy_new function:

struct wiphy *wiphy_new(const struct cfg80211_ops *ops, int sizeof_priv);

As you can see, this function takes a struct cfg80211_ops *. This is how the driver tells cfg80211 which callbacks should be assigned to which functions — essentially struct cfg80211_ops is a big list of function pointers.

Next the driver registers the wiphy with cfg80211 using wiphy_register():

int wiphy_register(struct wiphy *wiphy);

passing it the newly created wiphy. As you might guess, there are also wiphy_unregister and wiphy_free functions to remove and clean up a wiphy.

The process is similar for drivers using mac80211, except that they will instead use ieee80211_alloc_hw(), passing it a struct ieee80211_ops *, and ieee80211_register_hw(). There are matching unregister and free functions here as well.

Of course, this would not work very well if the callbacks were the only means of communication: we also need a way for the driver to call into cfg80211/mac80211 when it has an event to report or information to share (such as when a frame is received). Unfortunately, the functions going the other way are not neatly collected into one struct, but they are all available in the cfg80211.h and mac80211.h header files, so you can see which functions are available. They are all nicely commented as well, so you can find out what they do and when to call them.

But where do I find…? A tour of the source code

If, like me, you are not very familiar with the Linux kernel source code, it can be difficult to find where all the relevant parts of it are. So here is a guide to where you will find the various parts of the wireless subsytem. Here, I’m going to use $linux to refer to the location of the Linux kernel source code.

• $linux/include/uapi/linux: Here you will find nl80211.h, which contains all the definitions needed to use the nl80211 netlink protocol.
• $linux/net/wireless: And here is nl80211.c, with the code that handles the kernel side of the nl80211 socket. In particular, here you will find the struct nla_policy for each attribute, which can be very helpful as the required data types are not always listed in the header file. The functions to handle each command/attribute will also show you the causes of the various errors you might receive, which can sometimes be a bit mystic.
• $linux/include/net: Here you will find cfg80211.h and mac80211.h, containing all the definitions needed for wireless drivers to use these modules.
• $linux/net/mac80211: This directory houses the mac80211 source code. In particular, main.c has the mac80211 implementations of all the ieee80211_ops, and cfg.c has the code to communicate between mac80211 and cfg80211
• $linux/drivers/net/wireless: In this directory we have the code for the various wireless card drivers, each in their own subdirectories (mostly). For instance, if we look in $linux/drivers/net/wireless/ath/ath10k, we will find the code for the ath10k driver. You can look in core.c to see the code that gets the module up and running (and shuts it down again), and mac.c contains the ath10k implementations of the ieee80211_ops callbacks, as well as struct ieee80211_ops ath10k_ops, listing which callbacks ath10k implements and which functions they are mapped to.

Further reading and helpful resources

• Linux wireless developer documentation
• Linux wireless mailing list Warning: Joining this list will result in a lot of patches in your inbox. Make sure to read past posts and/or lurk for a while before posting.
• Linux wireless IRC channel: #linux-wireless on irc.freenode.org
• Linux loadable kernel module HOWTO
• Netlink library (libnl) documentation