Compiling a Go app, that uses Cgo, to run on an Edison

[TLDR - Use a Docker container, install the 32-bit compiler chain and 32-bit dev libraries on the build machine, include the flags to enable CGO, set the target GOARCH, set ldflags to statically link]

So, originally I thought I'd be able to easily get my (trivial) Go app to cross-compile for the Edison (which is running a 32-bit linux variant) but I was quickly disabused of that notion. My app uses the `gopacket` library, which in turn uses C bindings (Cgo) to `libpcap-dev` to do the actual packet capture.

I had originally thought it was just a matter of adding "GOOS=linux GOARCH=386" to compile from my OSX box to target a 32-bit linux binary. It works fine for most apps, BUT it doesn't work for apps that are using Cgo. Ah, just forgot the "CGO_ENABLED=1" flag, right? Nope. That causes all sorts of different errors. Googling/Stack Overflow didn't really turn up anything helpful, but there's *probably* a way to do it. (there were a few projects out there, including gonative that seemed promising, but `gonative` only addresses Cgo-enabled versions of the stdlib packages, not if your project uses Cgo)

Rather than dig into the intricacies of cross-compiling on a Mac, I just pivoted to using a Docker container. I couldn't find an official Yocto linux container prebuilt, which is what the Edison runs, so I just went with a standard Ubuntu image.

root@ubuntu-docker-instance# go build listen.go 

Sweet, compiled. All set, that was easy. Let's just check the binary to make sure all is good with the world.

root@ubuntu-docker-instance# file listen
listen: ELF 64-bit LSB  executable, x86-64, version 1 (SYSV), dynamically linked (uses shared libs), for GNU/Linux 2.6.24, BuildID[sha1]=11b30b279fd6580392e72bac70a5be034e12b2a7, not stripped

Oops, dynamically linked. Let's fix that by setting the correct ld flags, and check it again.

root@ubuntu-docker-instance# go build --ldflags '-extldflags "-static"' listen.go 
root@ubuntu-docker-instance# file listen
listen: ELF 64-bit LSB  executable, x86-64, version 1 (GNU/Linux), statically linked, for GNU/Linux 2.6.24, BuildID[sha1]=8a5a63b0151c2cef02399d091ea7339b2ca1d30b, not stripped

Ok, statically linked now but it's (still) 64-bit. The Edison is 32-bit, which means we'll have to fix that... should be cake, just use the "GOARCH" flag...

root@ubuntu-docker-instance/util# GOARCH=386 go build --ldflags '-extldflags "-static"' identify.go 
# command-line-arguments
./identify.go:19: undefined: pcap.FindAllDevs
./identify.go:23: undefined: pcap.OpenLive
./identify.go:23: undefined: pcap.BlockForever

What?! Oh, yes, need to tell the compiler about the Cgo code... (CGO_ENABLED=1)

root@ubuntu-docker-instance/util# GOARCH=386 CGO_ENABLED=1 go build --ldflags '-extldflags "-static"' identify.go 
# runtime/cgo
In file included from /usr/include/errno.h:28:0,
                 from /usr/local/go/src/runtime/cgo/cgo.go:50:
/usr/include/features.h:374:25: fatal error: sys/cdefs.h: No such file or directory
 #  include 
                         ^
compilation terminated.

C'mon... now what? Some quick googling turns up the fact that you need the 32-bit gcc stuff, since the host architecture is 64-bit.

root@ubuntu-docker-instance/util# apt-get install libx32gcc-4.8-dev libc6-dev-i386

root@ubuntu-docker-instance/util# GOARCH=386 CGO_ENABLED=1 go build --ldflags '-extldflags "-static"' identify.go 
# github.com/google/gopacket/pcap
/usr/bin/ld: cannot find -lpcap
collect2: error: ld returned 1 exit status

Now what? Oh, although I have the 32-bit compiler chain, I DON'T have the 32-bit version of libpcap-dev. Let's fix that.

root@ubuntu-docker-instance/util# apt-get install libpcap-dev:i386
Reading package lists... Done
Building dependency tree       
Reading state information... Done
Package libpcap-dev:i386 is not available, but is referred to by another package.
This may mean that the package is missing, has been obsoleted, or
is only available from another source

E: Package 'libpcap-dev:i386' has no installation candidate

Huh? furious googling ensues... Ok, add the i386 architecture and try again...

root@ubuntu-docker-instance/util# sudo dpkg --add-architecture i386
root@ubuntu-docker-instance/util# sudo apt-get update
root@ubuntu-docker-instance/util# apt-get install libpcap0.8-dev:i386

NOW we're cooking with fire. Let's give it one more try...

root@ubuntu-docker-instance/util# GOARCH=386 CGO_ENABLED=1 go build --ldflags '-extldflags "-static"' identify.go 

Looks promising, it's a 32-bit statically compiled binary at this point. No obvious errors.

identify: ELF 32-bit LSB executable, Intel 80386, version 1 (GNU/Linux), statically linked, for GNU/Linux 2.6.24, not stripped

SCP the binary onto the Edison and test it...

root@edison:~# ./identify -inf=wlan0
2015/09/04 14:11:02 Starting up...
2015/09/04 14:11:03 Listening for ARP packets on en0 to identify dash button.
Press the dash button and watch the screen. You will want the 'Source MAC'
2015/09/04 14:11:09 ARP packet, Source MAC[74:75:48:a4:59:a8], Destination MAC[ff:ff:ff:ff:ff:ff]
2015/09/04 14:11:24 ARP packet, Source MAC[74:75:48:29:a8:7c], Destination MAC[ff:ff:ff:ff:ff:ff]

SUCCESS!!!

Playing with Amazon Dash buttons

Amazon is experimenting with wireless buttons (for $5 each) called Dash Buttons. The intended usage seem to be for Prime members to stick the button near a product that needs periodic refilling, e.g. laundry soap, so that when you're almost out of the product, you press the button and it (almost) magically reorders for you.

These buttons are nifty little pieces of technology, especially for $5. Someone has already figured out how to use them for something other their intended usage - read his post on Medium, it's very well written. He has source code in Python for accomplishing the hack.

I wanted to take a slightly different route to accomplishing the same type of thing. I have a bunch of hardware laying around that would be fun to play around with, including an Intel Edison board. I could write my code in Go, cross-compile it on my Mac, and just SCP it onto the Edison. Sounded like a fun evening or two, so that's what I did and threw it up on github.

Tested the code on OSX and it works. Attempted to compile it for Linux, so I can deploy it onto the Edison, and compilation fails...

$ GOOS=linux GOARCH=386 go build identify.go
# command-line-arguments
./identify.go:19: undefined: pcap.FindAllDevs
./identify.go:23: undefined: pcap.OpenLive
./identify.go:23: undefined: pcap.BlockForever

I'm pretty sure it uses CGO and there are some details about cross-compiling that I'll need to figure out... but that will be for another post.

Getting RStudio to work with RWeka and the previous java 1.8 problem

So, from the previous post, installing Java 1.6 and recompiling rJava is enough to get everything working from a terminal window. But, if you start up RStudio and try to use RWeka (or other libraries using JNI, probably) you'll see an error message similar to:

Error : .onLoad failed in loadNamespace() for 'rJava', details:
  call: dyn.load(file, DLLpath = DLLpath, ...)
  error: unable to load shared object '/usr/local/Cellar/r/3.2.0_1/R.framework/Versions/3.2/Resources/library/rJava/libs/rJava.so':
  dlopen(/usr/local/Cellar/r/3.2.0_1/R.framework/Versions/3.2/Resources/library/rJava/libs/rJava.so, 6): Library not loaded: @rpath/libjvm.dylib
  Referenced from: /usr/local/Cellar/r/3.2.0_1/R.framework/Versions/3.2/Resources/library/rJava/libs/rJava.so
  Reason: image not found
Error: package or namespace load failed for ‘RWeka’

A workaround is to start RStudio from a terminal window by running the following line:

LD_LIBRARY_PATH=$(/usr/libexec/java_home)/jre/lib/server: open -a RStudio

Getting various R packages to work with Java 1.8 and rJava on OSX

If you attempt to install various R packages (e.g. RWeka) on OSX using a Java 1.8 runtime, you're likely to run into the following error message (even though you have a perfectly good installation of Java):

"No Java runtime present, requesting install.
ERROR: loading failed"

There are a ton of threads out there about this problem, but it seems to boil down to this unclosed (and untouched since 2014-01) JDK issue -> https://bugs.openjdk.java.net/browse/JDK-7131356

The "solution" can be found on the last comment on this rJava github issue: https://github.com/s-u/rJava/issues/37

1. install JDK 1.6 from Apple (http://support.apple.com/kb/DL1572)
2. open a new terminal window and make sure you're still using 1.8 - "java -version"
3. run "sudo R CMD javareconf -n" from a terminal window
4. start up R and install rJava again from source - install.packages("rJava", type="source")

Continuous integration: Rmd to md to html

I'm taking one of the Coursera courses right now on Reproducible Research and the first assignment requires you to build an Rmd file and convert it to html (using knitr). They do a really good job showing how to use RStudio to accomplish this task, but what if you don't want to bother with RStudio?

Here's (https://gist.github.com/slowteetoe/7b8426f567ac3df1d8f9) a simple bash script that watches an Rmd file. When you save that file, it runs knitr to convert it to markdown, and then converts the markdown to html and opens it in a browser... sorta a continuous integration for R markdown.

Couple caveats: you'll need to install the Ruby gem 'kicker', the r packages 'knitr' and 'markdown', and put this script somewhere in your path. It will also have to be made executable, 'chmod 755 knitr'. Also, it opens a different browser tab each time - but I couldn't figure out how to change this behavior and it was worth it (to me at least).

R annoyances

R seems to be, hmm, quirky, from what I've seen of it so far... For instance, ddply has different behavior using the American vs. British spellings of "summarize". Seriously?!

> g <- ddply(m, c("QGroup","Income.Group"), summarize, All = length(CountryCode))
Error: argument "by" is missing, with no default

> g <- ddply(m, c("QGroup","Income.Group"), summarise, All = length(CountryCode))

Yes, that's right - "summarise" works fine but "summarize" apparently takes different arguments.

Trouble installing jpeg R package on OSX?

I'm taking the Coursera "Getting and Cleaning Data" class in R and one of the quizzes requires you to use the jpeg package. The R package install was failing for some reason though:

install.packages("jpeg")

In file included from read.c:1:
./rjcommon.h:11:10: fatal error: 'jpeglib.h' file not found
#include 
         ^

I use homebrew, so I tried the usual...

brew install jpeg
Warning: jpeg-8d already installed

brew unlink jpeg && brew link jpeg

But that didn't fix the issue. The solution was to install the XCode command line tools with:

xcode-select --install

(As a side note, this is my development machine so I already had XCode command line tools but for whatever reason I had to install them again, maybe an XCode upgrade?)