Compiling a Go app, that uses Cgo, to run on an Edison

[TLDR - Use a Docker container, install the 32-bit compiler chain and 32-bit dev libraries on the build machine, include the flags to enable CGO, set the target GOARCH, set ldflags to statically link]

So, originally I thought I'd be able to easily get my (trivial) Go app to cross-compile for the Edison (which is running a 32-bit linux variant) but I was quickly disabused of that notion. My app uses the `gopacket` library, which in turn uses C bindings (Cgo) to `libpcap-dev` to do the actual packet capture.

I had originally thought it was just a matter of adding "GOOS=linux GOARCH=386" to compile from my OSX box to target a 32-bit linux binary. It works fine for most apps, BUT it doesn't work for apps that are using Cgo. Ah, just forgot the "CGO_ENABLED=1" flag, right? Nope. That causes all sorts of different errors. Googling/Stack Overflow didn't really turn up anything helpful, but there's *probably* a way to do it. (there were a few projects out there, including gonative that seemed promising, but `gonative` only addresses Cgo-enabled versions of the stdlib packages, not if your project uses Cgo)

Rather than dig into the intricacies of cross-compiling on a Mac, I just pivoted to using a Docker container. I couldn't find an official Yocto linux container prebuilt, which is what the Edison runs, so I just went with a standard Ubuntu image.

root@ubuntu-docker-instance# go build listen.go 

Sweet, compiled. All set, that was easy. Let's just check the binary to make sure all is good with the world.

root@ubuntu-docker-instance# file listen
listen: ELF 64-bit LSB  executable, x86-64, version 1 (SYSV), dynamically linked (uses shared libs), for GNU/Linux 2.6.24, BuildID[sha1]=11b30b279fd6580392e72bac70a5be034e12b2a7, not stripped

Oops, dynamically linked. Let's fix that by setting the correct ld flags, and check it again.

root@ubuntu-docker-instance# go build --ldflags '-extldflags "-static"' listen.go 
root@ubuntu-docker-instance# file listen
listen: ELF 64-bit LSB  executable, x86-64, version 1 (GNU/Linux), statically linked, for GNU/Linux 2.6.24, BuildID[sha1]=8a5a63b0151c2cef02399d091ea7339b2ca1d30b, not stripped

Ok, statically linked now but it's (still) 64-bit. The Edison is 32-bit, which means we'll have to fix that... should be cake, just use the "GOARCH" flag...

root@ubuntu-docker-instance/util# GOARCH=386 go build --ldflags '-extldflags "-static"' identify.go 
# command-line-arguments
./identify.go:19: undefined: pcap.FindAllDevs
./identify.go:23: undefined: pcap.OpenLive
./identify.go:23: undefined: pcap.BlockForever

What?! Oh, yes, need to tell the compiler about the Cgo code... (CGO_ENABLED=1)

root@ubuntu-docker-instance/util# GOARCH=386 CGO_ENABLED=1 go build --ldflags '-extldflags "-static"' identify.go 
# runtime/cgo
In file included from /usr/include/errno.h:28:0,
                 from /usr/local/go/src/runtime/cgo/cgo.go:50:
/usr/include/features.h:374:25: fatal error: sys/cdefs.h: No such file or directory
 #  include 
                         ^
compilation terminated.

C'mon... now what? Some quick googling turns up the fact that you need the 32-bit gcc stuff, since the host architecture is 64-bit.

root@ubuntu-docker-instance/util# apt-get install libx32gcc-4.8-dev libc6-dev-i386

root@ubuntu-docker-instance/util# GOARCH=386 CGO_ENABLED=1 go build --ldflags '-extldflags "-static"' identify.go 
# github.com/google/gopacket/pcap
/usr/bin/ld: cannot find -lpcap
collect2: error: ld returned 1 exit status

Now what? Oh, although I have the 32-bit compiler chain, I DON'T have the 32-bit version of libpcap-dev. Let's fix that.

root@ubuntu-docker-instance/util# apt-get install libpcap-dev:i386
Reading package lists... Done
Building dependency tree       
Reading state information... Done
Package libpcap-dev:i386 is not available, but is referred to by another package.
This may mean that the package is missing, has been obsoleted, or
is only available from another source

E: Package 'libpcap-dev:i386' has no installation candidate

Huh? furious googling ensues... Ok, add the i386 architecture and try again...

root@ubuntu-docker-instance/util# sudo dpkg --add-architecture i386
root@ubuntu-docker-instance/util# sudo apt-get update
root@ubuntu-docker-instance/util# apt-get install libpcap0.8-dev:i386

NOW we're cooking with fire. Let's give it one more try...

root@ubuntu-docker-instance/util# GOARCH=386 CGO_ENABLED=1 go build --ldflags '-extldflags "-static"' identify.go 

Looks promising, it's a 32-bit statically compiled binary at this point. No obvious errors.

identify: ELF 32-bit LSB executable, Intel 80386, version 1 (GNU/Linux), statically linked, for GNU/Linux 2.6.24, not stripped

SCP the binary onto the Edison and test it...

root@edison:~# ./identify -inf=wlan0
2015/09/04 14:11:02 Starting up...
2015/09/04 14:11:03 Listening for ARP packets on en0 to identify dash button.
Press the dash button and watch the screen. You will want the 'Source MAC'
2015/09/04 14:11:09 ARP packet, Source MAC[74:75:48:a4:59:a8], Destination MAC[ff:ff:ff:ff:ff:ff]
2015/09/04 14:11:24 ARP packet, Source MAC[74:75:48:29:a8:7c], Destination MAC[ff:ff:ff:ff:ff:ff]

SUCCESS!!!