Summary — What Could Go Wrong?
- This is a big data project (Price Guide found here): data points & estimates are not individually inspected; rather I collect auction data from a major eBay consigner and then use a statistical model to try to extrapolate the recent value of a card based on its past values, which are adjusted for price trends (in the broader market and for the particular player / subset) since the card was listed
- Though always incomplete, I am always chipping away to develop strategies to reduce lost accuracy & errors:
- Data points could be off due to mistakes in auction listings, which would hurt estimate accuracy
- Data points could be off due to occurrences of “shill bidding” that can occur on auction platforms, which would hurt estimate accuracy
- Data points could be off due to unknown, irregular, characteristics of a particular auction listing, that are not easily detected based on the data that my web scraper collects (e.g., if a card is graded by a service I do not know about, then I have no way to account for the effect of this premium service has on the card’s final sale price), which would hurt estimate accuracy
- When certain players or subsets do not go for sale often, then the model cannot be precise in updating the price trends for their particular cards. Rather, an average market trend is used, which results in lost accuracy — if, for example, the player has fallen out of favor, or the subset was originally perked-up by a fad that has since died… then the values would be overstated if there were not many sales since the decline in price
- Estimates will be off when a particular card has few recent data points & its price movements behave in an idiosyncratic matter; certain cards might abide their own logic in terms of price movements
- There may well be other factors I cannot account for…
How the Estimates Work
Roughly, this website is a project, based out of academic curiosity, which tries to estimate the present value of a card based on data about cards from the past (realistically, the “present value” should be construed as the “recent value” of the card — card prices move around really fast these days, and I only update every 2-3 weeks). I “scrap” auction sales data off of eBay, for one particular seller: COMC.com. Because COMC is just one seller, that cannot possibly be selling every notable card with any sort of regularity, I must use a statistical model to try and estimate what their cards, sold in the past, would be worth in the present.
Largely, this is done by adjusting the price of cards based on general trends in the market. To estimate the current value of a card that sold a month ago, not much adjustment will likely be needed. But, let us say that the card market has been crashing for the last year (which it has been, as I write this at the beginning of 2023); in such a case, if a particular card’s most recent sale was a year ago…. Well, then that card’s value should probably be adjusted downward in reflection of the downward slide of prices in the market.
Of course, this is quite crude. By this logic, if the market is down 50% since the card’s last sale, then the card’s value gets “penalized” by half. But, a player that has long-term injury, or did terrible things in their non-sporting life, is probably down more than 50% — in contrast, a player that had a career year is probably not down the full 50% — heck, maybe they are even up, defying the broader market trend.
To partially redress this, my model then adds more “individualized” adjustors for cards. So long as there are sufficient data points, then the statistical model will try to adjust for trends in the values of more nuance features about the card (such as the player or subset to which it belongs). So, maybe a particular Connor McDavid card only has one data point from a year ago… but some of his other cards are up over that time… well, then the model will put an upward pressure on the price for that card from a year ago, to try and better reflect the player’s price movements.
The models try to do these sorts of adjustments based on the player, the set (in fact, I use the subset) and the year the card was produced. Note that the adjustments only occur if a minimum thresholds of card sales are made for the particular category: so, for example, I cannot adjust the Connor McDavid card’s value for the ‘player effect’ if there have only been a few of this player’s cards sold over the year (this is obviously a silly example — McDavid has lots of cards selling). So, for example, if I have not witnessed at least 5 price movements in my data set for a particular player, then I do not try to make an adjustment based on the specific player’s price trends. Likewise for the subset that cards are from.
Let me finish this section by noting: I am not appraising cards one-by-one. Rather, I scrap data from Ebay, and then build a statistical model that will try to estimate what cards are worth based on a combination of what they sold for before and by consider what we might learn about this card’s value based on what has been happening to other cards values. This occurs through an automated process… Unsurprisingly, I am not checking all 100,000+ estimates individually.
So… in terms of our estimates, what can go wrong? Well, I sense errors can come from two places.
So… What Can Go Wrong?
So… in terms of our estimates, what can go wrong? Well, I sense errors can come from two places.
1. The Data
First, I cannot presume the data I am collecting will be perfect, nor that I can perfect it.
I use a web scraper to collect realized auction prices on eBay from the consignment service COMC.com (Ebay User: comc_consignment)
COMC does two things: they write a title for each auction and they identify in their description the “identity” of each card based on their internal-library-system (each distinct card gets its own “code” in the library). The title is designed for marketing purposed (to attract eyeballs), whereas the “identity” is for organizational purposes. My models use the “identity” feature, but it is important to note that choices COMC makes in the title can affect the auction results in ways that hurt my model’s accuracy.
So, the title might read with a bunch of key words, like “2015 Upper Deck Young Guns Connor McDavid Rookie RC,” whereas the description will follow the internal logic of their library’s labeling system, e.g., “2015-16 Upper Deck – [Base] #201 – Young Guns – Connor McDavid”.
Herein, two issues (…that I can think of…) might arise. COMC might make a mistake in the title of the listing, resulting in the card going for less than it should. This would get carried into our estimates for the card’s value (which is not based on the eBay title — we do not detect if mistakes were made in the title). OR, the title might be fine, but COMC might mislabel the “bin” to which the card belongs in their library (again, this is not found in the auction’s title, but the auction’s description). The card may then sell for its market value, but our estimate will assign that value to the incorrect “identity.”
Less technical, but two other issues might arise. First, market manipulations. There are definitely a lot of shill bidders out there, who might push up the value of a card, without any real intent to buy it. Obviously, this would result in our models having a data point that overstates the value and, thereby, our models would likely overstate the value. We do not have any great techniques, as of now, to identify shill bids in our models. Second, sometimes there are weird qualifiers for an auction listing that I struggle to incorporate into the models. For example, I try to remove cards that have been graded… but this is hard to do when I don’t know the grading service — I cannot write into my computer program a condition to remove such items, since I do not even know the grader name in the first place. There are also after-market autographs. Some people get cards signed by the player, but my models may not pick-up on the fact that this card, which is not normally autographed, has been autographed thereby increasing its value… I try to remove these sort of items, so they don’t affect estimates, but some may get through.
2. The Statistical Modeling: Trying to Extrapolate Over Time
Okay, this should be obvious, but I will put it out there: the card market is super complex. There are so many individual cards. Many try to find ways to guess what prices movements will happen to what cards — but there is just way too much going on to capture every effect that happens.
More specifically, there a few difficulties to elaborate upon….
First, we might have a card in our data base from a sale a long long time ago. Our model is going to try to predict today’s price based on assumptions about what price movements in other cards might tell us about this card. So, if we don’t have a recent sale of Connor McDavid’s Young Guns rookie card, then we might adjust the price of it upwards (or downwards) if his other cards, for which we do have recent values, have been going up (or down). So, here we have a difficulty: what if there is something particular about this card — some sort of outside appeal — that just makes it different from other cards. It might follow its own logic in pricing and, if so, then our model won’t capture that “X-factor” in the price trend. So, to the extent we have thin data on a card, and its price trends do their own things, distinct from cards that are similar to it, then we are “shit out of luck.”
Second, maybe you do feel — as I do — that most of the time there should be patterns whereby similar cards should have similar price movements. So, if a player is trending up, that should affect all their cards … or if a certain subset is becoming popular, then all players of that subset should be getting a little bit of a tailwind…. To the extent this is true, then my models should definitely be helping to extrapolate old auction data to today’s value. But, what if there is simply other stuff that I cannot fit into my models? I can put “player” into the model, and “subset” and the “year” (to capture potential effects distinct to, lets say, vintage cards)… but there are so many other things that I just cannot get into the data. Maybe Croatians are really starting to love hockey and players from that country are going up… Well, that is really hard to capture.
Note that I think many troubles pricing-out cards is a hybrid of the two effects above. Cards mostly do follow patterns based on their “vintage,” their player and the (sub)set to which they belong. Upwards or downwards pressures in prices for the categories of a card will likely be shared with other cards in said category. But, cards do have a hint of individuality. Trends do not affect all cards equally. Good times for a player will likely cause a boom for their limited run rookie cards, but only slightly create an uptick in their more mass produced inserts. The effect of playing 10% better might be a 10% boost in some of the players cards, but a 50% boost in others. Obviously, this sort of thing will be tough to capture, unless I have enough data on every individual cards — which, of course, often I don’t have that. Without enough data, I need to use the general price trend for the player… but the price trend due to the player is an average effect, that in fact varies based on other characteristics of those cards that feature the player.
Third, sometimes the data is just to thin to adjust values for a card based on important criteria, such as the player. Let’s say a player is a star but then does something really terrible. Maybe the card value crashes and so people don’t auction his cards. Well, then, my models are going to struggle to update the value of that player’s cards, because there is no data to do so. This is, obviously, a challenge. If the declining value of a card means fewer listings of it, then that causes my models to lack data needed to update to the lower pricing.