KolibriFX.com

For the last two months, focus has been on building Milestone 1 of our trading platform, based on the feedback we received from the London visit. We have not been blogging about this, mostly because we’ve had no big news from week to week — the development has been slow and gradual.

However, we feel it’s about time we give curious readers a sense of what we’ve been up to, and what we’re planning in the next few weeks and months.

Alpha Release

Towards the end of July, we will publish a release of Milestone 1 to a very small set of test users. This alpha release features:

• a visual development environment for creating trading models from ready-made parts by dragging components onto a canvas, configuring the components, wiring them together, connecting up market data feed and configuring order routing, all in a visual way;
• a simple rule language for expressing temporal patterns on prices and other data streams;
• a simple, configurable risk manager that imposes hard limits the exposure of a trading model;
• a small library of ready-made components for building models;
• live connectivity to a broker for paper trading; and
• support for simple backtesting on a limited set of tick data.

It should come as no surprise that we’ve packaged all of this as a web site. It is this web site that will go live (but remain closed to the limited group of testers for now).

The Road Ahead

We start by opening up the alpha release to our technical advisors. After a few weeks, when we’ve worked out most of the obvious bloopers found by them, we will gradually widen the flock of testers to include traders who have been involved in the process from the start.

We hope and expect this period of testing to help settle the design of the trading environment. As the various part settle, we’ll be posting some screenshots and in particular, we will offer more details on our domain-specific trading language.

At some point around the turn of the calendar year, we expect to open up a public beta. We offer no guarantees as to the exact time frame — we feel it’s more important to deliver something reasonably solid, than to deliver something reasonably on time.

Until next time, you are more than invited to follow us on Twitter: @kolibrifx.

We can happily report that the exciting times are not over for us: Last week, we went to London to present the  zeroth milestone of KolibriFX. Milestone 0 is mostly a proof-of-concept. No parts of it can be said to be anywhere near production quality. However, it served us well in gathering valuable feedback.

Originally, we had not planned on demoing anything this early in the development process. We’ve only been working on this rather ambitious project for the better part of three months, and, as we’ve already indicated, a lot of our time  has been spent learning and prototyping with new or somewhat unfamiliar (at least to us) technologies. (We found it reasonable to conduct some due diligence before committing to one technology or technique over the other, e.g. Scala over Java or an agent-based trading architecture over a stream-based one.)

During our first three months, we have enjoyed expert advice from friends with considerable technical know-how, and been very happy with this. However, when we received a call from a sizable UK hedge fund (who read our blog) to show what we’re up to, we found it a great opportunity to also get feedback from the domain experts — the traders. And since we were going abroad anyway, we lined up additional meetings with brokers, quants, more traders and technologists, to get their feedback as well.

The main motivation for us would be to ensure that we’re targeting our future users as accurately as we can. One important aspect of this is letting others help us in prioritizing what we should focus on next. Another is trying to keep a very open mind: it is very easy to keep your own preconceptions when listening to other people’s feedback. To improve the chances for relevant feedback, we put together a very short presentation of where we want to be, followed by a “product” demo of where we are now. We then invited our audience to suggest ways from here to there.

The demo contained the highlights from milestone 0:

• KolibriFX as an Eclipse 4.0-based (pre-release) Rich Client Platform (RCP) application.
• A text editor for our trading language, with syntax highlighting, realtime syntax errors detection and a little bit of semantic error detection.
• An extremely rough sketch of where we want to go with the graphical visualization of our trading langage, implemented with JavaScript and HTML5 Canvas.
• A post-trade analytics screen where a graph of the cumulative PnL as an interactive chart, implemented with the Google Charts Annotated Timeline widget.
• The full edit-compile-run-analyze cycle (but with a detour into a separate Eclipse IDE instance, since we haven’t hooked our compiler into the RCP yet).
• Mockups of other parts of the system, such as the trading and configuration screens.

The process of explaining our motivation and goals, then showing the demo, made it all the much easier for our “reviewers” to hammer in their points about where we fall short. Having an actual, running “product” to talk about, focused the discussions a lot, and seemed to make people all the more intense in their feedback. In fact, we were pleasantly surprised at the level of enthusiasm and opinions the demo managed to evoke. Suggestions for improvements were given freely and at a very high pace in all of our meetings.

We knew before we went, that our presentation would be extremely rough, especially when it comes to usability. Most of the UI had been slapped together in the week before the demo, and that’s likely why most of the feedback was related to the user experience (or lack thereof). In brief, the most important take home messages were:

• Simplify and streamline the product; what you showed us is too complicated and cluttered.
• Continue to work on hiding away as much technical and domain-specific complexity as possible.

In retrospect, this isn’t a surprise at all. We’d already anticipated some of this feedback (we do play the Devil’s advocate on each other a lot). However, hearing this from other people who know what they want, who’s been down some of these roads before, who seem very much to know what they’re talking about, gives us a solid affirmation that simplification should be our top priority now.

Based on the feedback, we must conclude that Takuya, our fictitious user, is probably a lot more technologically sophisticated and forgiving of technical details than what our eventual, real user will be. He might very soon be joined by a less techy, more financy guy. Possibly with a tie.

The month of April has, with the interruption of Easter, been entirely focused on two things: (1) the design and implementation of our trading language, based largely on the domain analysis we did in March, and (2), a thorough survey of suitable UI frameworks, combined with building working prototypes for the difficult most difficult UI elements using the most promising framework candidates. But the UI story will have to wait until a later post.

A few days ago, we came to one of the most satisfying turning points in any language implementation: we finally managed to compile, package, deploy and execute the simplest of test models. The test model consists of one agent. The agent listens to prices from the market data stream, performs some trivial computations on these, then dumps the result to our debug console. Not exactly rocket science, but entirely rewarding nonetheless.

We now have all the pieces in place for the play button to work for simple trading models: write your model and agents in our programming editor. Hit play. Watch it run. Be happier, and more productive (and fitter too!).

Behind the scenes, the following takes place:

• The text is parsed and an abstract syntax tree (AST) is built.
• The AST is type checked.
• The AST is given to our compiler, which produces a Scala program.
• We use the Scala compiler API to create class-files.
• We zip up the class files in a jar-file.
• We boot up our trading platform.
• We hand the jar-file to the trading platform
• The trading platform reflects over the jar, instantiates the model in a fresh classloader, deploys it, hooks up all the necessary wiring, and executes it.
• After 10 seconds, the platform closes down

Provided the model contains no errors, all of this happens fully automatically. If errors (type errors, internal compiler errors) arise, the process stops early with more or less useful information in the debug console. For debugging purposes, we keep the jar-file around, but all of this can happen entirely in memory, leaving no traces on the disk. Might be useful in environments where a file system is generally unavailable, such as in, say, a web browser.

We’ve been deeply immersed in domain analysis for our upcoming trading language for the past month. One important part of this analysis has been the vocabulary and language of trading.

As a companion to our domain analysis, Karl Trygve did a survey of the state of the art in writing computer programs in the English language.

So as we are progressing in building our product I’d like to give you some insight into our working methodology here at KolibriFX. We, like most entrepreneurs out there, are super enthusiastic about our product. And all great products grow out of such enthusiasm. But it also makes us prone to the pitfall of cluttering it with all sorts of cool features that will look great in the brochure, but is shying away from what the customer really needs. This entry will show how we address that challenge.

Like many others in the programming community we were inspired by Alan Cooper‘s “The Inmates are Running the Asylum” and the concept of personas. In a nutshell, Cooper’s mantra is that in order to build great products you need to visualize your end-user during the entire innovation cycle. If you don’t have a customer, make one up. And we are not just talking about making up names here, but fully fledged bios including everything from education to what car they drive. In our case, we created Takuya, a 40-year old electrical engineer trading margin FX for his own account out of the suburbs of Tokyo.

Takuya is married and a father of two. His 10-year career at Mitsubishi Electrical corp. ended after he decided there was too much travelling involved and he wanted to spend more time around his family. Consequently he set up a trading operation from his home-office hoping to make enough profit to sustain his current lifestyle. His strategies are largely a combination of technical indicators and statistical analyses, and his current toolbox consists of MetaTrader4, MatLab and various online web forums such as dailyfx.com. He knows his way around spreadsheet applications and statistical software, but he is not a programmer and has never heard about threading or race conditions or any other complicated programming issues. He knows about so called Expert Advisors (EAs) that are basic automated trading models that can be downloaded or bought from knowledgeable programmers and plugged into MetaTrader. Although he likes the idea of removing the emotional aspect from his trading and extending trading hours with automated execution, he has decided against using them because (1) he hasn’t found one that covers his strategies yet; (2) having a custom EA built is expensive; and (3) maybe most importantly: he wouldn’t be comfortable being dependent on an outsider if a model needed calibration due to shifting market conditions.

So that’s our guy. And our task is to address his three concerns by giving him a new set of tools that will help him to automate his strategies without external help and keep him in full control over his trading.  Whenever we face problems or questions around interaction design we always ask: What would Takuya do? Does he really need this feature? We have found it remarkably helpful to bring Takuya into all sorts of discussions, even when making decisions about core infrastructure that he will never see.  We have his full bio taped on the walls all around the office (even in the toilet!) to ensure we never forget who we are working for!

Almost a month has passed by since we last blogged. I am now glad to report that this is not because we’re dead, but because we’ve been extremely busy. The last four weeks have been all about domain analysis. For casual readers without a software engineering background, “domain analysis” is “the process of analyzing related software systems in a domain to find their common and variable parts”[1]. More specifically, we’ve tried to break down the somewhat complicated field of forex trading, into self-contained entities such as instruments, currency pairs and zones, prices, sides, as well as time series and windows. For each entity, we’ve worked through what you can do with it — such as compare two prices, ask how close a price is to the nearest big figure, add pips to a price, and so forth. We’ve had to spend quite some time on how the various entities (also called abstractions) relate to each other — in the domain of forex, what is (or should be) the difference between an instrument and a currency pair ?

Along with the domain analysis, we’ve prototyped many of the abstractions in Scala.  This has allowed us to gain some familiarity with each of the abstractions, and measure how they stack up to the expectations of our domain expert, Einar.

Perhaps the most challenging aspect we encountered during the analysis has been how to deal with time. Our goal is to express trading rules in a succinct and readable way. As it happens, most non-trivial trading rules incorporate the notion of time, such as

when the correlation between the price and the index deviates by more than X within 3 minutes,

, or

trade when the 10 minute moving average crosses the 20 minute moving average from below.

I cannot yet show you the exact code for solving these two examples, because we’ve not settled on a syntax for our DSL yet. However, I can sketch out in broad terms what the solution would look like. I’ll do this for the second example.

The sentence above gives rise to four separate agents, 10 minute moving average, crosses,  20 minute moving average and trade. If you are programmatically inclined, you can think of an agent as an “active object”, running in its own thread, receiving and sending messages asynchronously. At any point in time, only one thread may be executing inside a given agent, so all agent code you write is familiar and sequential — you don’t deal with multi-threading and thread-safety inside your agent logic.

The four agents are wired so that both 10 minute moving average and 20 minute moving average receive a live price feed from the market in the form of a stream of price messages. They send their messages to the cross agent, which in turn sends its messages to the trade agent.

The logic inside the moving average agents takes care of the actual computing of the moving average. This is rather easy, since all they have to do is look at an x minute time window of the latest prices, and update the moving average every time a new price enters the window, i.e. whenever they receive a new price on their price feed. Every time a moving average is computed, the agents emit a message containing the computed result to cross. The crossing agent is only concerned with looking at the historical plot of the moving averages. Whenever the 10 minute moving average plot crosses (from below) the 20 minute moving average, cross will emit a signal to the trader agent. The trader agent will then do a trade — the concrete details of which are left out of our simple example, but could be on the form buy 1M EURUSD.

Using the English language as a syntax for programming has been attempted by many, and looks intriguing on paper. It does however open up a can of worms on the topic of ambiguity. We have been experimenting with various ideas and looking at existing work in the field, and will provide our conclusions in a later post. The gist of it is that rather than having a pure natural-language like syntax, we are leaning towards a solution where the DSL has two parts: (1)  a less-programmatic syntax for expressing complex patterns (as seen above), and (2) a more ALGOL-like syntax for library code and computations.

In the next few weeks, we’ll be settling on a first cut of the textual syntax of our language. This will set the stage for a lot of interesting examples on this blog in the weeks and months to come.

Hi all,

On the technology front we are still working on getting the runtime environment going. A key point in our strategy is to leverage on high-quality Open Source components where possible, so we have spent a lot of time performance-testing various frameworks (some mentioned in earlier posts) and are looking to spend at least another week benchmarking before making final decisions on what to take on board (we will report our findings here). On the business side, we have established relationships with a few brokerages already, and are setting up connectivity with at least one external UAT-environment this week. It will be exciting to see trades go through our pipeline for the first time.

Back to work! See you again soon!

This week, we’ve been in “DOS mode” (for the uninitiated, DOS-mode is when you put on your headset, crank up the volume, open a terminal window and enter into the world of the command prompt). The work has concentrated on two fronts: getting our basic runtime up and running, and making simple trading models in Scala using our framework for benchmarking purposes.

Technically, there isn’t anything terribly exciting to report on, yet. Our runtime hosts an arbitrary number of trading agents that can talk to each other asynchronously, using an actors-based message dispatcher. The runtime itself is connected to the world around it through two ports: input and output. The input port receives pricing events and the output port emits trading events.

The agents are somewhat restricted in how they may organize themselves, so as to ensure a properly directed dataflow, in the style of, say, Dryad. The aim of such a model is ensure that we have no crippling loops in our event flow, and that parallelism is manageable. Each agent believes it is a single-threaded process when it handles a given incoming event (and is, in fact, prohibited from spawning any threads on its own).

We’ve looked at and tried a couple of actor frameworks for Scala and Java to see how much we could avoid coding ourselves. The most impressive Scala-based framework appears to be Akka, but it’s not performant nor stable enough for our needs at the moment. The jury is still out on which, if any, we’ll employ.

On the modeling front, we’re having pretty good success with Esper for so-called complex event processing. It performs pretty well. However, with some pattern matching expressions, we get a heap space blow-up when subjecting it to non-trivially sized event streams. A topic of investigation for next week.

All in all, we’re left with more questions than answers, despite good progress. But such is the way of DOS.

Hi all,

We can happily report that we are making progress according to plan!

In addition to getting actual coding done, we’ve spent a lot of time discussing important design issues and conducting surveys on available open source technologies. The project has three major parts: the runtime environment and supporting infrastructure; the domain specific language; and the web tools. We’re scheduling our progress in that order, basically hoping to

• have a basic distributed core and a demo trading model running by the end of this month
• spending most of our time after that working on the domain specific language and its support libraries
• release a proof-of-concept version and a very basic web toolkit by August.

We plan to release the core runtime and the domain language under an Open Source licence, and will let you know once we open up our source repository.

On the technology front, we’ve settled on the JVM as our runtime. We are aware that this impacts the potential for ultra-low latency models, but this segment is presently somewhat out of scope for us. Overall, the JVM brings more to the table than we lose: Scala as our workhorse implementation language, live migration of code across network nodes, an slew of existing, solid support libraries for everything under the moon, just to mention some benefits. We’re already taking advantage of the last property: we plugged in Esper to solve our complex event processing needs. Esper’s capabilities fit very well with our needs, and our initial performance tests are acceptable, though we’ll conduct more thorough testing in the weeks to come.