Monday, December 7, 2015

Is go [golang] Suitable for Algorithm Deployment?

Before I tell a story about how I ported the front-end of my C-based Reed Solomon algorithm demo to go [golang], and post the results, I will answer the obvious.

Yes, go is suitable for algorithm deployment.  Enterprises deploy algorithms in Python, so clearly go is suitable: its syntax is a blend of C and Python, and it is a relatively fast compiled language.  Moreover, it's easy to inline existing C code and libraries, which is what we show below.  The performance penalty of calling C from go is minimal.

Linking and Inlining C (Good)

So, I went through the trouble of porting the demo front-end into go directly.  The algorithmic backend remained as a static C library (librs.a).  Including these elements into a go package is one part makefile and one part inline coding.  go requires that you put the C code and build instructions in comments above the import "C" directive, but that's about it.  This is nice.  I can't understate how much better this is that using pymodule, JNI, or even JNA.


Handling Data that C Would Put on the Heap (OK)

go isn't explicit about heap and stack.  It has garbage collection, and so forth.  However, it's basically no different than using malloc() apart from not requiring the mating free() calls.  Not really a big deal for an expert programmer.



Strict Typing (A Real Pain)

Good data algorithms are not built around the idea of strict types, they are built around the idea of transforming a pile of bytes into another pile of bytes.  I understand why the go designers kept-out soft typing (and in fact made go very pedantic about typing), but in certain cases it's just silly.  Perhaps the best example of the silliness, here, go's inability to use arithmetic conditionals.


Performance Results, and Indications of Call Overhead

go has some overhead when it calls C, but not much.  I ran these demos a bunch of times, at different times of the day, and the results were consistent -- there's a subtle overhead of using go to call C, even when using "unsafe" pointers when calling C.

You'll notice that the actual timed part of the loop is just one call, and it doesn't pass-back any data.  So what we are observing in the minimal call overhead.  It amounts to a few percent (4-7%).  It's absolutely great that it's so easy to do "unsafe" calls to C, though.


$ ./c-rsdemo 220 16 30
TESTING MESSAGE:
Encoding message...
0.262620 s
114.078601 MB/s

Decoding Message...
0.362215 s
81.608471 MB/s
0/124833 packets identified as uncorrectable


Hooray, all errors corrected!

$ ./go-rsdemo 220 16 30
TESTING MESSAGE:
Encoding message...
0.281255 s
105.555642 MB/s

Decoding Message...
0.383857 s
78.051895 MB/s
0/124833 packets identified as uncorrectable


Hooray, all errors corrected!