Things to avoid while using Golang plugins

We are planning to open-source our project. There are some code about authorization logic that is highly specific to the company and we want to provide a way to not include that part of the code. But we still need to provide that same functionality. And while doing that we don’t want anyone to re-compile all the code just to make it work with their implementation of that specific part.

Our codebase is in Go which I love to work with. And while searching for possibilities, we came across golang-plugins. With golang plugins, you can import functions and variables from another file that was built via go build -buildmode=plugin command.

Gopher by Renée French (CC-BY 3.0)

We quickly got started and prepared a prototype implementation for our need. But, while working on that we came across some problems. Here are some of the problems we faced and ways to avoid them:

1) Different go versions

Both plugin implementation and the main application must be built with the exact same version of the Go toolchain. Depending on your GO version you will get an error like this:

panic: plugin.Open("simpleuser.plugin"): plugins must be built with the same version of the Go toolchain as the main application

OR

panic: plugin.Open(“simpleuser.plugin”): plugin was built with a different version of package github.com/alperkose/golangplugins/user

Workaround: None.

Since the code provided by plugin will run in the same process space with the main code, the compiled binary should be 100% compatible with the main application.

2) Different GOPATH’s

Both plugin implementation and the main application must be built with the exact same GOPATH. The error you will get is:

panic: plugin.Open(“differentgopath.plugin”): plugin was built with a different version of package github.com/alperkose/golangplugins/user

Workaround: use the same GOPATH (GOPATH in official docker image is /go) The issue https://github.com/golang/go/issues/19233 is opened for a similar situation.

Example code: https://github.com/alperkose/golangplugins

3) Using vendor folder

This seems a bit related to #2 but if you use vendor folder in either your plugin and/or your main application, you will get a very weird error:

panic: interface conversion: plugin.Symbol is func() user.Provider, not func() user.Provider

If you look closely, you will see that expectation func() user.Provider and the actual signature func() user.Provider are the same. This was a very confusing error but in all versions since 1.8 it exists.

Workaround: copy all your folders in your vendor folder to gopath while building your binary. This is a very dirty workaround and you need to do this for your plugin and your main application. If either one of these binaries are build with vendor folder, you cannot use the golang-plugin solution.

In our case, we were copying the folders in vendor and then we removed vendor folder during build phase. This was done in a docker image, so our local folder structure remained unchanged while during development.

...
RUN cp -r vendor/* $GOPATH/src && rm -rf vendor/
...

The issue https://github.com/golang/go/issues/18827 is opened for the same situation.

Example code: https://github.com/alperkose/golangplugins

4) Different versions of common dependencies

Any dependency you have in your plugin should be the same version with the dependencies in your main application.

Again, since the code provided by plugin will run in the same process space with the main code, the compiled binary should be 100% compatible with the main application. When you compile your binaries, the 3rd party packages are also compiled in your binary, but if there is a different version of the same function in your process space, your binary will panic since the compiled versions are not the same.

Workaround: use package managers and ensure the dependencies are the same version

You can find comments about this issue here: https://github.com/whiteboxio/flow/issues/3

5) Building static binaries

You cannot compile plugins into static binaries. I love static binaries since it removes the requirement of having a base image in your docker images. Using docker scratch image provides a minimal docker image and reduces one big dependency. When I tried to build a static binary out of plugin, I failed and you can find why in this article.

Workaround: None. You need to compile with CGO and if you’re using docker, you need a base image in your Dockerfile.

Conclusions

I think that golang-plugins are not a mature solution. They force your plugin implementation to be highly-coupled with your main application. The end-result is very brittle and hard to maintain even if you control the plugins and the main application. The overhead will be much higher if the author of the plugin does not have any control over the main application.

All of these issues pushed us to consider alternatives, in the end we’ve chosen to use hashicorp’s plugin package. It is based on RPC communication and provides us enough flexibility, however, it has it’s own limitations which are easier to overcome than golang-plugins

Code

You can find the code to test issues #2 and #3 here: https://github.com/alperkose/golangplugins