Juggling & gluing with Extinct
Erlang/OTP is considered by its insiders as a fantastic coordination language. That means it makes a great glue for systems that need to juggle many moving parts: parts written in languages, other than Erlang.
It’s good at juggling because a concurrency model was built right into Erlang from day one. They chose asynchronous message passing as the means to perform such trickery, which makes OTP a great tool for gluing polyglot systems into one.
Interportability is something Erlangers should care about for the obvious fact that our community is small, and libraries can be quite mature in other communities that are teeming with people and activity.
Say, for example, there’s a powerful Machine Learning tool written in Python. Perhaps hundreds of programmers have contributed to years of development of this tool. Needless to say, it would be silly to rewrite its equivalent in Erlang. Rather, by leveraging one of the many Erlang interportability options, we can just interface with this Python library and get on with our day.
What is Extinct?
Extinct is a new tool written in C that digests OSC messages and spits out that same data in Erlang’s ETF (external term format). It’s a tool designed to mitigate the headache involved with connecting programs together. Extinct relies on Erlang’s ‘ei’ library to encode the ETF and also uses various open-source code to digest incoming OSC messages.
(OSC is a simple yet expressive protocol which can be used like one would use JSON and XML. The OSC protocol can be learned about here.)
The goal with Extinct is to treat your OTP
hub system as a first-class citizen, and it’s
quite accommodating for that reason.
After your hub system executes the code of another
language, by simply piping the yields of those
programs through Extinct, your hub system gobbles
up that data in ETF. Your OTP hub system
doesn’t need to do any decoding — it just needs
to do a single
presto, the data is in your hands in easy-to-use
The public repository for Extinct is here:
make, one can run programs that
output valid OSC messages and simply pipe the
output to the
extinct executable. The executable
can be found in ‘priv/bin.’ Here’s an example to
drive the point further:
./runs-then-outputs-osc.sh | path/to/priv/bin/extinct
extinct output is a binary ETF representation
of the original OSC data, and with that, you’re half
the way there.
The Erlang side
It’s assumed that anyone needing to use Extinct has a good grasp of Erlang, so with that, just a few words about the approach on the Erlang half of things.
If you take a look at ‘src/extinct.erl,’ you’ll be greeted with some familiar-looking textbook Erlang that is ready to digest ETF via Extinct using erl_interface — a methodology that mixes Erlang Ports with Erlang’s ‘ei’ library.
The Erlang Extinct source files (located in ‘src’) is a good place to peruse. Since everyone’s needs are different, the code in there is just for reference.
You can run an Erlang Extinct sample via
make sample after the initial
make has been
Things to consider
This project is in its infancy, so be prepared for additional functionality as time goes by. For example, at the time of this writing, the only data types on the Erlang side are binaries, strings, integers and floats. This will improve over the coming months to include more data types.
Again, the OSC protocol can be learned about here. The next section is a breakdown of the types that Extinct supports.
Supported incoming OSC types
- b: binary blob
- f: float
- d: double
- i: int32
- h: int64
- s: string
- m: midi
- t: timetag
- T: true
- F: false
- I: infinitum
- N: nil
This list is more or less complete.
Supported outgoing ETF types
This list is a work in progress.