Dark, I'm feeling Dark. Go ahead, bite me. |
The concept of dark bullion was first written about in analysis #2 where it was noticed from the SLV data that some of the bars appearing as 'new' inventory had actually been listed some time prior. In analysis #3 and #4, we progressed the investigation and now we know that bars can go dark and resurface, sometimes reappearing in different vaults (and we can track their movement from London to Australia, for example). Hidden, juicy secrets! Exactly what are those bars are doing when they are not represented on an ETF's bar list? While we will never know, at least we can describe what we can't see!!
Let's start with a premise. The "Dark Inventory" concept in commodities is not a new one ¹ but it is fair to say the area is difficult to study because of the general lack of quality information. However, the silver and gold inventories provide an interesting opportunity to track inventory because (conveniently) each bar has a serial number. So ... what is available to find, and how do we find it?
First, the mechanic. At the point a bar goes missing we truly have no idea what happened to it. But if the exact same bar signature shows up in another bar list document at a later date, we can safely argue that the bar existed the entire time and was not (for example) melted down into coins the day after it disappeared. If the same signature shows up later in the same vault then we could put forward a case for it having been in the vault the entire time it was showing up in any bar list (not precise, but plausible and probable). The longer a bar stays AWOL, the less certain we can be about whether it will show again. Regardless; with enough data points we can construct a basic narrative around an individual bar - here's what it looks like graphically:
In 2011, Bron Suchecki described the possibility that if we hoard enough data we may be able to get a glimpse of the hidden secret lives of the LBMA bullion bar in its natural habitat. The SLV bar list provides a great opportunity to see this effect since these days the documents are issued nearly daily, and the data volume is substantial. A new bar 'appearing' in the source documents can only be one of the following:
Dark Bullion - A bar previously known and registered, re-appearing after an absence (dark bullion). Technically this bar ceases to be dark bullion once it appears back on the bar list, but once the two points are known, we can assign a value for how long it has been 'dark'. Scientific name is H. Organis Nemesis (named so after a precious metals blogger famous for penning many words in criticism of how SLV can add so much inventory so quickly).
New Bullion - A bar which appears and is completely new to our database. This is a big limit of reading error since we can't actually know whether the bar was already registered within other parts of the LBMA bullion banking system - all we know is we never had a record with that signature, after all we don't have EVERY document issued by every fund. But while we will never know the origins of the bar, we can at least get an approximation of it's time in the Bullion Banking system by flagging it with a moving-reference-point sub-classification. New bullion can be considered either Hot (Sprotticus Hyperus - bars still warm from the refinery produced within 12 months) or Cold (Sorosicus Fossilus - bars which have been produced ages ago but are still around for some reason). Please also note that for the purposes of this classification, the actual age of the bar is a different measurement altogether - perhaps it needs a further qualifier. New bullion (to the ETF under study) may also come from another ETF, an effect called 'vault jumping' which we've already explored.
The dark bullion label is temperamental because we can only call it dark if we know for certain it reappears later. It means the picture gets better over longer periods of time, but is not so useful on the short horizon. While we wait for that essential second piece of data we do require using the following classification (which is final until proven temporary).
Lost Bullion - A bar which we have at least one reference point for, but has since vanished and we have no idea of it's whereabouts. Lost bullion presents a challenge because we have to write it off once we don't see if for a certain length of time. Additionally, if it suddenly reappears then we have to correct the record and say that it had been dark all that time, rather than being lost. This is a challenge to display graphically, but at least we can gradate the severity - a bar that's been missing for 1 month should be represented differently compared with a 12-month long disappearance **. Collecting more data sources over a longer time period should statistically reduce the amount of 'lost bullion', especially if we assume large portions of the bullion is likely to stay in the bullion banking system once put into bar form.
And a definition which we'll explore later:
These definitions form the nucleus of discussion around ETF inventory. It's worth noting these definitions are ETF-centric; they don't describe things which cannot be seen or properly measured in the data we study, like 'off-market' bullion. The definitions also work well together with the other definitions of 'vault jumpers'.
Sticky Bullion - (Beirus Wixus Conspirasus) A bar which doesn't appear to move, once added to the fund inventory. Sticky Bullion is the polar opposite of dark bullion since it always stays on the same ledger. If a bar moves from the list it ceases to be Sticky and must be classified using one of the classifications above. The identification of Sticky Bullion is highly susceptible to the 'changing signatures' limit of reading error (described below).
Graphs
Putting all those together, here's my current graphs for SLV dark bullion, you can click on the image for a better view and details. The information is dense, so read further for explanations about what is represented here.
The chart above provides a graphical view of the dark bullion effect in SLV. The black band running through the middle is dark bullion - bars which we know for certain have been removed, but for which we also have confirmed results that it returns at a later time. It's easy to see some basic patterns here:
- The 'camel hump' formation in the first part of the chart (prior to April 2011) represents a period of bullion being removed from SLV, but with the benefit of over three years worth of data collection, we can see that most of that silver actually stayed in the vault during that time.
- From August 2011 through to Feb 2013, the inventory appears reasonably stable, with a smooth slope upwards. What I mean by 'stable' is that Dark + Visible Bullion had very little volatility, despite the visible inventory being somewhat volatile.
- The large uptick in late February 2013 seems to introduce a large amount of new bullion to the inventory. Unless the signatures have changed, it doesn't appear to have come from the lost bullion amounts, however again if you draw a few straight lines then the stability appears to continue.
The volume (of the black band) effectively represents 'known outstanding dark bullion', and it tapers off towards 2013 because we don't have that '2nd point of reference' to prove that some of the lost bars are actually dark. That will come in time, however we can go directly to our next anomaly. I mentioned in my last article how some bars have a tendency to 'disappear and then reappear'. The same thing happens with SLV, here are the figures.
With such a large sample size, I suppose it is natural to have some bars which get shifted around quite a few times. While it's possible that there are some data anomalies which account for this, generally this requires an exact signature match to occur. To put it in context, the data shows that over the last three years, 350 bars have been removed from SLV, then added back, then removed again, then added back, removed, added, removed, added again, removed once more, added, removed, added, removed, added, removed, added, removed again, and finally removed once more. Insane, right? Especially since I've got the data to prove this is a real effect and not just a figment of my imagination.
This result suggests there is some inventory which is 'more prone to being relocated' than other bullion. Fortunately we can use the 'number of times removed' to tell us more information.
In this chart, the black band running through indicates confirmed dark bullion, but we've added more information which shows the dark bullion being reintegrated into visible bullion, or being 'lost' forever. To get this chart, we tallied the number of times a bar has been 'dark' and then identified its position in visible or lost inventory. This gives a better sense of the size of the bullion repository that the bullion banks are using, and again the amount of inventory in use is reasonably stable. I'll be happy to explain this a bit more in the comments or in private correspondence.
Limitations of This Study
Very important - I already know the graph doesn't show a highly precise picture of what it is trying to represent, it's like a fuzzy TV reception. Over time, we can tweak the resolution, but it's important to list the flaws of this current data interpretation:
Very important - I already know the graph doesn't show a highly precise picture of what it is trying to represent, it's like a fuzzy TV reception. Over time, we can tweak the resolution, but it's important to list the flaws of this current data interpretation:
New or Old? In this study there is currently no distinction between hot new bullion and cold new bullion this is because we have not yet launched into a detailed study on the serial sequences, but it's a distinction which will improve the overall picture so we'll get to it soon. For now, all new bars where we haven't seen the signature before, will show as hot new bullion (regardless of age), except in the case of serial numbers where the year is part of the prefix. We hope to fix this in later studies.
Graph Height. The SLV inventory is so massive that for this graph, 1 pixel = approximately 300 bars. This seems insane because a lot of detail is being glossed over, so just be aware you are viewing things at an incredible height (we do have the capacity to report the same information at a bar-by-bar level but we don't currently have a handle on how to present that much information on a single screen).
Starting points. This is the picture that we have with the current set of data. It might look completely different if we had all of the original document issues since the ETF started. In other words, we can only chart the data points we do have access to, and the dark bullion graph looks slightly different each time we get new information. We do have one SLV file from 2007 but it is encrypted which prevents extraction.
Changing Signatures. If anyone adjusts a bar record then our signature doesn't match and the computer will think we have new bullion and lost bullion. We must always operate under the assumption that the funds will always be making adjustments, and also that their data is not perfect. For now, my current estimate for the effect (on accuracy) resulting from changing serial numbers is ±5% which is large but only affects the 'lost' and 'added' figures. The dark bullion identification is not affected by this - i.e. it requires an exactly matching signature to show up. What it means is that the amount of dark bullion would/should be higher than what we've calculated here, once we've cleaned the data more.
Duplicate Signatures. SLV has so many records that perfect internal duplicates are present, even on some high complexity index' items. The presence of duplicates will sometimes affect the tallies since in some aggregations this detail is lost, however we're talking very small percentages here, literally a fraction of a percent. So although this is present, the discrepancy is generally not relevant. For an analysis of duplicate signatures in SLV, check out http://about.ag/SLV which provides these in great detail.
Vault Jumpers Data Incomplete Index. In this graph we don't show Vault Jumpers, i.e. where we know the bar has been found in another document once it exits SLV. In the data we have identified positive matches, at time of writing in May 2013, around 4 million oz of silver originally in SLV have ended up in : BullionVault Silver (London), Perth Mint Pool Allocated Silver, iShares Physical Silver ETC (SSLN). Please note that some of the data from the other funds is not 100% up-to-date and the other funds don't issue daily documents so showing it on this chart would be difficult. Suffice to say at time of writing, the 4moz represents only about 1% of the total height of the chart (and accounts for about 2% of the lost bullion).
Lost Bullion. The lost bullion can take many forms, it might have been made into coins, it might be held privately, it might have gone to other ETF's that we don't have data for, it might be signatures that were simply wrong which have since been corrected (so it seems like the bar ceases to exist). Because of the definition, we don't have much data to work with. The figures for lost bullion shouldn't be read into that much, however it is a very large amount so even allowing for a fair percentage of limit of reading error, it represents a substantial amount of silver bullion.
Interpretations and Implications (Mine)
Courtesy of the dark bullion effect, there are more bars in the 'system' than what formally appears on the SLV listing. Basically I say bollocks to the stories about silver shortages, since by May 2013 there are many thousands of bars seen in the data but no longer accounted for (in addition to the visible inventory). The 'Dark Bullion' aspect of the gold and silver market has not had much attention because it was previously unmeasurable. My primary goal (and reason for writing), is to prove outright that the market is more complicated than most expect and observe. In my opinion, most of the popular precious metals sites, ranging from Harvey, TFMetals, Maquire, KWN and even GATA, will need to recalibrate some of their previous ETF inventory analysis, i.e discussions about ETF 'shedding inventory' can only be properly discussed using the definitions I've written about here. The classifications have been refined over the last 24 months. No doubt we can fine-tune or add to the overall taxonomy, but these will always be at the core.
* Gratuitous "They Might Be Giants" reference http://tmbw.net/wiki/Lyrics:Reprehensible
¹ Isabella Kaminska has penned a few articles which may be relevant to the exploration of Dark Inventory in several articles here, here and here (2012) but to be honest, (a) dark inventory in market usage is beyond my current level of understanding, and (b) I also think the ETF narrative has several special mechanics, not currently explored fully; for example dark inventory in copper is different to dark inventory in gold. In reference to what she writes about gold, is there a chance she's accidentally describing dark bullion?
** In determining the write-off period for 'lost bullion', Bron Suchecki has suggested a formula using the average & standard deviation of the number of days a bar is dark. We can calculate this information and we hope to use it on some future graphs.
I welcome open discussion on these definitions; receiving proper critique would be great. I do request that any comments be kept (1) civilized, and (2) related to the primary article content. The research took over 8 months to prepare and I consider it my most important article to date, as such I may relocate your content if I think it contravenes my two very basic requests, thanks all.
I would like to publicly acknowledge the help of several contributors who helped me arrive at this level of analysis: Bron Suchecki of the Perth Mint in Western Australia, Mark Hynes, and the team from http://about.ag/SLV. Additionally thank-you to all of our regular Screwtape Files readership who have contributed at different times with questions and observations about my area of study.
The database data producing the dark bullion graphs has over 250 million rows in it. It would not have been possible to process this amount of data without the 'columnstore index' feature of SQL Server 2012 (xVelocity). My only hope is that Microsoft make this a standard feature for their entire range of database software and not just the Enterprise SKU.
13 comments:
"this bizarre niche topic"
A niche topic at the moment, but I think this work on dark bullion will come to be something that is covered by most commentators, as we accumulate more data and the fuzzy picture firms up.
"could put forward a case for it having been in the vault the entire time"
It is not so much whether it has been in the same vault, more important is the fact that the bar was not melted and used, that is was obviously in existence during the whole time it was not visible.
"New bullion can be considered ... Cold .. may also come from another ETF, an effect called 'vault jumping'"
Once we have a better handle on serial numbers' relationship to date of manufacture (that will be hard to establish) then that will reveal a whole lot of new dark bullion.
Vault jumpers to me are also a classification of dark bullion. OK, the bar was in another vault/location, but again we know it existed up to time A and then it reappears at time B.
For me the relevant scope of focus should at least be a location, eg London, rather than specific ETFs or vaults.
"Sticky Bullion"
Interesting to me is any bullion which leaves an ETF and then immediately reappears in another ETF. Another form of sticky bullion I think is those bars who have a high number of appearances, ie those 'more prone to being relocated'. Certainly anything removed/reappearing more than 10 times has to be pretty sticky.
Thanks Bron, those are good definitions.
You're right, amalgamating the rest of the silver data should reveal a better industry-wide picture. There are over 700,000 unique silver bar signatures that we know of; the only requirement is to craft a normalizing mechanic for the ETF records where we only have partial appearances - for example some bar lists are issued yearly, quarterly and weekly so the graph would have to allow for those 'gaps' (the SLV data is relatively clean because we get to apply a daily comparison). The best picture may also come from merging all records because it would smooth out the inter-vault movements as well as inter-country (e.g. London to Perth).
I agree with your sticky bullion breakdown - the definition then becomes 'likely to stay in [area]', e.g. likely to be added/removed, likely to stay in vault, likely to stay in London, Zurich, etc.
By the way, I have realized for one of the next studies that we can also do individual snapshots - e.g. take a snapshot of bars added from a certain sample, flag them with color and then see where they end up (like injecting dye). This would allow visibility of specific inventory 'sets' and in theory allow us to spot unusual movements. As some know I'm still hoping to use all this data to catch out the bullion banks - the historical data is all there (3½ years worth), it's just a matter of asking the right questions.
How does this compare to GLD eft?? Does the gold "puke" indicator become ireviliant do to the accurance of dark bullion?? Do you see GLD bar flow inventory differ from slv bar flow inventory as in GLD is draining and slv is not??
@answer2me, the pattern for GLD is similar, i.e. with a dark band running across, but with less indication of a stable slope.
There are several artefacts on the GLD chart worth noting. When there was a sharpish drawdown at the end of August 2011, within that trough, the chart shows most of that missing bullion was dark and returned later.
It also shows a consistent band of inventory which is 'dark affected' like my 2nd graph above, which continues right up to our current figures at May 2013. However, it's worth noting that in the current figures they have removed all the previously dark bars ... i.e. seems to be getting down to the metal.
This is a bit hard to describe without showing the GLD chart itself - if there is enough interest then I may publish it. Regarding the (Lance Lewis) Puke, I don't actually know. I am in the process of trying to correlate figures to discover whether anything significant is happening with regards to bar movements at the time when a 'puke' is called. Hope that helps?
Great series. Very well done. I wouldn't get my hopes up on the PM sites, though. They are probably already constructing your effigy at TFMetals right now.
Warren,
Fuckin eh! It would be extremely interesting to know if there is some kind of correlation when a "puke" is called. With the amout of tons that appear to be draining from GLD I wonder if any of those bars will show up in vaults outside of GLD. It looks like we could be going sub-1000 in GLD tonage, since the onslaught of pukes we have lost about 300 tons. Thats a lot of serial numbers that are bound to turn up somewhere sooner or later. It would be interesting to run your data looking at specific dates that dark bullion came back into inventory. Pull a few dates and look at amounts. I wonder what the variance would be, is it always the same amount added back, is there a pattern ect...
Just for giggles, try this:
http://projects.wsj.com/games/thefederator/?mg=inert-wsj
Hasn't all the Gold been shipped to Russia and China by now?
Warren,
OUTSTANDING WORK!
Yes, all caps because I shouted it out with enthusiasm and appreciation :) i will read it again later and much closer. I already have several questions that I want to ask, but will wait till I have reread later today.
Milamber
Hello, fascinating analysis, thank you.
On the graph height problem, perhaps you could employ a real-time mouse following magnifying glass GUI?
Like Apple do with their Retina screen ad?
I have no idea, but is this code suitable for the job?
By the way, this is not my area of expertise, just a thought on how I might attempt to tackle the problem of elegantly displaying the fine detail in a large data set.
@Reality Show, thanks. Could indeed create what you describe (since my day job is software design) ... but the challenge runs deeper than just displaying 225 million pixels in a single image.
In a nutshell, any chart has to highlight detail relevant to what's being shown, but the human brain can't take in large amounts of detail so we have to summarize it, but then we lose some detail. Being able to see individual bar movements would be a bit like looking at a 5-minute chart for three years worth of data - would get quickly bogged down.
I like to think we're developing a new 'ETF Inventory Science', and hopefully we'll be able to filter out what aspects and patterns are generally interesting and which ones are not. Feedback from readers like yourself will ultimately help calibrate the areas which are worth looking at (hence why I write).
Hope that makes sense?
Hi Warren,
I had imagined the magnifying solution might well allow you to peer into, say, 5 minute resolution on a multi-month price chart, but yes, I think I see the wider issue of data comprehension now.
I'll be very keen to see if the current drain out of GLD turns out to be more of a Dark Bullion event. Almost everybody on the planet seems to mis-understand and mis-report action in the GLD, I'd love to know what is actually going on.
Anyways, thanks, you seem to be uncovering a deeper perspective.
@Reality Show, I too am interested in the current draw-down. I guess in about 6-12 months we'll know either way. If it helps, the average of 30% bullion returned on an 'Add Event' in 2013 shows that at least SOME is going dark (it would be fair to extrapolate from previous results that 30% of the current bullion removed is still in the vault).
I just had my attention brought to this software for charting (link), which might be able to provide bits of what you described (e.g. automatic zoom support). You can bet that I'll explore it in more detail.
Post a Comment