This post is in multiple parts:

I’ve been spending some time trying to learn a bit about Machine Learning recently, and as part of that, I wanted currency rate data.

The data is publicly available direct from the European Central Bank (ECB), and I’ll set out in a series of posts how I went about fetching, parsing and turning it into a microservice that can fetch live or historic data.

Euro Rates

The ECB provides this information in XML, CSV, PDF, and through an RSS feed.

For the live data feed, the XML looks like this…

<?xml version="1.0" encoding="UTF-8"?>
<gesmes:Envelope xmlns:gesmes="http://www.gesmes.org/xml/2002-08-01" xmlns="http://www.ecb.int/vocabulary/2002-08-01/eurofxref">
	<gesmes:subject>Reference rates</gesmes:subject>
	<gesmes:Sender>
		<gesmes:name>European Central Bank</gesmes:name>
	</gesmes:Sender>
	<Cube>
		<Cube time='2017-05-22'>
			<Cube currency='USD' rate='1.1243'/>
			<Cube currency='ZAR' rate='14.8198'/>
		</Cube>
	</Cube>
</gesmes:Envelope>

I’ve cut the list of currencies for clarity.

The naming of the elements isn’t great (Cube) but it’s clear from this that I have something like this…

date: "Date"
  currency: "USD" rate: "amount
  currency: "ZAR" rate: "amount"

i.e. for a given date, I have list of currencies and their amount (relative to the Euro).

Structs

In Go, this can be described by a struct type:

// EuroRate is the data for a given currency on a specific date.
type EuroRate struct {
	Currency      string
	Rate          string
	ReferenceDate string
}

Note that I’m opting to keep the Rate as a string, there’s no benefit in losing precision by translating to a float value here, there is a Go currency library available here but there’s no need to convert to a floating point value.

This makes it easy to model the contents of the XML as a map.

// EuroRates represents the rate for a specific date for several currencies.
type EuroRates map[string]*EuroRate

It’s trivial to add a method to a map in Go, so to simplify the lookup for clients I can do this:

// GetRateForCurrency returns the EuroRate for the named currency or nil if
// there is no data for that currency.
func (r EuroRates) GetRateForCurrency(c string) *EuroRate {
	return r[c]
}

Parsing the XML

Parsing XML in Go can be fairly simple, the encoding/xml package can do most of the hard work for you, but it relies on having slightly more structured data than is available in the ECB XML. All those Cube entries mean that I have to resort to custom decoding.

Starting with a test

I almost always start with a test, and given the unknowns around exactly how I was going to parse this, I started with something fairly simple.

The fixture file I downloaded had 31 currencies in it.

func TestParseDailyRates(t *testing.T) {
	f, err := os.Open("testdata/eurofxref-daily.xml")
	if err != nil {
		t.Fatal(err)
	}
	defer f.Close()

	rates, err := parse(f)

	if err != nil {
		t.Fatal(err)
	}
	if len(rates) != 31 {
		t.Fatalf("incorrect number of rates: got %d, wanted %d", len(rates), 31)
	}
}

I downloaded a sample of the daily data, and put it into a testdata directory below my code, something useful to remember is that the Go test-runner changes to the directory of the test file, so paths to fixture data can be relative.

Also, Go will ignore testdata directories when scanning for test files.

This test follows the classic Arrange, Act, Assert format for tests, it opens the fixture file (checks for errors) and then, acts by calling the parse function, then finally, asserts that I parsed the right number of elements from the test file.