In(transparency) at the Nairobi Coffee Exchange?
In the past one and a half years, I have been closely monitoring transactions at the Nairobi Coffee Exchange (NCE). One major improvement I have noted from the NCE is that they now have a website which is a big improvement. This helps to fulfil one of the NCE's mandate of fostering transparency in the coffee market.
What's more is that the NCE has been posting the reports of its market transactions after each of their Tuesday auctions. catalogues prepared by marketers are also made available. What is even more impressive is the fact that the NCE makes transaction data available in spreadsheet format which renders it useful for working with and carrying out analysis. There are lots of improvements that can be made but at this stage We should all celebrate the steps towards increased transparency. This is because reports were hitherto only sent out by email to a group of privileged people.
On this post I will focus on the quality and usability of the transaction data that is shared. This is the data that we have used to compile our coffee directory as well as our prototype coffee market prices which you can find on this website.
I consider myself to have above average intellectual capabilities but it took me a few spreadsheets to figure out what the transaction reports I was receiving were trying to convey. In the following sections I present the main issues I faced.
Incompleteness of the reports
Whereas the NCE goes a long way to provide transaction information, one of the main things that is missing is an index of terms and IDs that are used in their reports. This makes it difficult for anyone who has no background or insight in the coffee business to understand what is happening. The transaction report for instance provides truncated names of coffee buyers e.g. "Nairobi Java Cof" instead of "Nairobi Java Coffee House Ltd". This is however, manageable because one can use the partial information to research in other resources to find out the full names as we have done on this website. The huge Bermuda Triangle at the NCE, for me at least, is really the absence of the identity of the coffee cooperatives and producers.
Variations in how marketers prepare catalogues made it difficult to draw patterns. The names of cooperatives/estates are especially an issue because some share the exact same name which makes it difficult to disambiguate them. The spelling of some cooperatives/estates at times follows a non-standard and native spelling. The reference numbers as well follow different naming structures e.g. XCE001F01 vs XCE01F1 or CB0098 vs CB.0098. Marketers also seem to have a free hand in the use of spaces, hyphens and slashes.
Inaccuracy of data
Data entry mistakes such as "O"s instead of zeros top the list of inaccuracies. These were however, the easiest to fix. Typos on the other hand are also common and take a bit more time to fix.
Unintegrated and incoherent reports
The task of integrating the data prepared by individual marketers appears to be one that the NCE has a lot of work to do on. Resolving the data silo problem would imply changing systems that each marketer is comfortable dealing with. At the NCE, the presentation of reports on their website within individual and nested folders as standalone "entities" makes analysis and getting the bigger picture difficult.
These problems can be attributed to the data silo problem caused by the distributed and unintegrated data management regime. This results in difficulty in tracing the provenance of coffee which also makes it difficult for coffee producers to track the transaction date, volume, grade and value of their coffee at the NCE. Lots of data is already available but it is difficult to know what is there. Inconsistencies also complicated automation through scheduled imports and joining of database tables. Lastly, and most important to me, the time taken to enter and clean data could surely be used for more productive work.
The good news
It is not all sad and grey when it comes to data management at the NCE. Most of the issues I have identified above can be attributed to the marketers who prepare the catalogues. The NCE itself seems to be having a data management system which can be seen by the consistent reporting and referential integrity of the agent IDs presented in the transaction spreadsheets which always match up with the buyer names in the transaction listing reports.
It is in the interest of the entire coffee value chain to push for increased transparency at the NCE rather than pushing for it to be done away with. Steps have been taken to improve the market. support and encouragement needs to be extended to ensure more transparency and traceability of the provenance, identity and stories of the producers behind the coffee. Based on what I have seen in the past year, I have the following recommendations for the NCE and any similar organisation with distributed responsibilities.
- Set up a master data management system that checks and integrates the catalogues from marketers
- Prepare standard reference lists of coffee producers with reference IDs that all marketers should use
- Define a standard naming convention/structure
- Apply data cleaning mechanisms to check and clean errors e.g.
- Consider researchers, farmers, non-specialist enthusiasts and the general public as potential end users of data