The efficient markets hypothesis may very well not be strictly true. But it cannot be false enough that Kalshi’s markets will be severely miscalibrated, can it? Given that so many participants are clearly recreational gamblers, could it be the case that the markets barely beat dumb base-rate baselines?

Kalshi Research released a paper on August 20 that showed its markets are well-calibrated, in line with what prediction market enthusiasts have preached for decades. Two days later, Tyler Cowen linked to a critique by X user @DeepDishEnjoyer which pointed out many flaws in the study, including that if you properly take base rates into account, the performance of the markets looks quite poor. People on X also noticed that Kalshi had to exclude the majority of its markets to get its shiniest results. If you include the entire market population, calibration looks significantly worse.

Now those critiques are surprising. We know, e.g., from Tetlock’s studies, that forecasting skill is a thing, and there is profit to be made by displaying it in the markets. For sure that will make the markets well-calibrated, won’t it?

I have downloaded data from every market in Kalshi’s history and replicated most results from the paper as well as the analysis made by the X posts I linked above. In this post I will share the mental model I arrived at. I will explain it first here then expand and share some charts.

To a first approximation, Kalshi is a gambling platform. Gamblers like noisy markets, so most of Kalshi is quite noisy and you should expect the overall performance of the platform to be worse than what you would get by excluding those for-degens markets. So you could say that excluding those markets is a form of cheating. I disagree.

Doing that exclusion was a sensible choice. I just don’t care as much about the calibration of contracts on dumb things like whether Trump will say “Barack Hussein Obama” in a speech as I care about forecasts for geopolitical events such as whether Zelenskyy and Putin will speak.

In fact I consider the paper fairly solid, if you read it with a dose of bounded distrust. The authors do have an incentive to make their platform look good, but their results are interesting and mostly track. They claim their markets are well-calibrated, and they actually are. Though they to a large extent leave aside the noisy markets that make up most of Kalshi, those are also fairly well-calibrated. Not that calibration is that high of a bar.

Still, contra @DeepDishEnjoyer’s findings, when you take base rates into account, their markets do show forecasting skill. Even noisy markets show skill, albeit less so. Some criticisms do survive. The paper has some inconsistencies in numbers and definitions which ideally should be corrected. But the paper is good.

In future posts, I will also share evidence that prediction markets are aggregating publicly available information reasonably well. In many domains, their performance looks at least as good as that of existing baselines. But that is not to say that they are providing a highly valuable service to society. They are not really taming public discourse, they are not great hedging instruments, and they certainly are not moving the needle on futarchy. My perspective is that in their current incarnation prediction markets really are mostly about gambling, which I personally care very little about.

Basically I spent the last month obsessing about Kalshi and Polymarket and have many things to say about them. This post is the first in the series, centered on Kalshi’s paper.

Kalshi as a gambling venue

There is this popular notion that prediction markets by and large function as gambling venues. That notion is true. On Kalshi, 96.0% of trading volume is either in crypto or sports, the number for Polymarket is 75.6%.

Just for fun, I asked Sol 5.61 to score markets on both platforms from 1 (“instrumental”) to 5 (“recreational”), with 3 representing “mixed or genuinely ambiguous”. Crypto and sports were almost entirely considered recreational, as one would expect: 99.98% of crypto and sports volume was recreational on Kalshi, 99.89% on Polymarket. Even disregarding crypto and sports, the bulk of the volume is still concentrated on recreational markets. Here is a breakdown of Kalshi’s 2026 listings:

Focusing on the slice of non-crypto, non-sports markets, 73.4% of the volume on Kalshi (29.2% on Polymarket) was still in recreational markets. One caveat worth making is that volume is subject to manipulation. Maybe crypto and sports do not dominate Kalshi all that much, maybe they just have way more wash trading.Sirolly et al. (2025) found that 25% of Polymarket’s volume over 3 years was likely wash trading, with the numbers for Sports and Elections being, respectively, 45% and 17%. So there is some evidence at least that wash trading biases the platform to appear more focused on recreational markets.

The largest category, Mentions, is about betting on, e.g., whether Trump will utter specific words. The third largest category is Entertainment, 23% of which is “first Super Bowl song”. The most “serious” categories are politics and elections.

Notably, climate and weather were overwhelmingly considered recreational, which I found strange since forecasting those is obviously economically important. Upon inspection, I side with the AI on this one. The structure of those markets makes sense if you interpret them as a gambling product that happens to use the weather as a source of noise. They don’t make much sense if the goal was for their prices to be used as forecasts.

For example, the largest series creates a daily market asking traders to forecast the highest temperature that will be recorded by a single station at Los Angeles International Airport (LAX) on a given day. LAX was the 13th busiest airport by passenger traffic last year, and high temperatures can affect aviation (as well as the need for air conditioning), so it isn’t as if anticipating that temperature had no economic value at all. But still, if the point was to offer forecasts for their usefulness, I would expect something more comprehensive than just daily high and low2 at a single spot for all of Los Angeles3.

Financials and commodities are other examples of categories that you would expect to be serious but whose volume is mostly concentrated in recreational markets. That is because a good number of those markets (example) are forecasting binary (up or down) changes in prices during brief horizons of 15-60 minutes. These are simply not good vehicles for hedging, or any other traditional financial use cases, but they do work well for gambling.

Of course you can always come up with stories about why a market is not recreational. Sports gambling can help the sport industry hedge its risks, some say. Similarly, most participants in “serious” markets can still be there for recreation. Election markets are “serious” because elections are important, but I assume they are as popular as they are because betting on them is fun. The estimates of the share of recreational markets I came up with, I would contend, tend to undercount the share of activity done for fun.

Betting was supposed to be a tax on bullshit, making the public debate more accurate by holding pundits accountable. Prediction markets instead became themselves mostly bullshit. Another proposed use case for them was to help organizations make better decisions, which could significantly increase world GDP. I was able to find no decision markets on either platform, and markets whose themes involve corporations are less about helping them operate more efficiently and more about speculating about newsworthy controversies (example).

Now to be fair, Kalshi and Polymarket do not directly claim to help with organizational decision making. Both do, however, claim a fact-checking role. And there are in fact people using them for this purpose. I will argue in a future essay that it is unlikely they are having a big impact disciplining discourse, however.

Manifold does have decision markets. Moreover, only 46.5% of its markets got labeled as recreational by the AI. That is kind of expected given that its small user base draws heavily from communities with many forecasting nerds.

Here is what the share of fun markets over time looks like:

Though it only jumped to being almost 100% recreational around October 2025, recreational markets clearly were a supermajority way before. I am not sure why crypto grew so much in 2026, but it may be due to the blockbuster 15-minute Bitcoin up/down series, launched in December 2025 and taking up 86% of crypto volume in August 2026.

If instead of aggregating on volume we simply count the number of markets, sports take a smaller share but it is still large, and recreational markets are still the overwhelming majority.

In any case, Kagan and Baiocchi, the authors of Kalshi’s paper, do exclude the noisiest markets from most of their results. Specifically, they exclude markets in the Sports and Mentions categories, as well as short-lived intraday4 markets. Interestingly, they did not exclude Crypto, although the intraday criterion does end up excluding most5 crypto markets anyway. Their criteria do not exclude any “serious” (scores 1 or 2 in the AI-graded scale) markets, but do exclude 85% of the recreational (scores 4 or 5) markets6. So it is approximately correct to say that they chose to mainly study non-recreational markets.

In theory, there are two effects at odds here, which naively would make it unclear whether excluding such markets is the sensible design. First, noise puts a bound on calibration: you can never do better than getting it right 50% of the time if you are predicting a coin flip. On the other hand, the markets called noisy are where most of the activity on the platform happens, and as the authors themselves establish, trading volume is correlated with better forecasts. In practice the first effect appears to dominate, recreational markets empirically display less skill forecasting the future, and I will show that later.

Have you checked the base rate?

Probabilistic forecasts, like those derived from prediction market prices, are commonly evaluated using the Brier score. The Brier score is calculated by taking a bunch of forecasts and averaging their error, so a higher value means the forecasts were worse. When people compete in a forecasting tournament, the winner is whoever ends with the lowest Brier score.

Though we can always say that a lower value is better, in a vacuum it is impossible to say if a given Brier score is good or bad. To decide that we must take into account the question population. As a first baseline, regardless of the precise forecasting tasks, a Brier score of 0.25 is always achievable by predicting 50% on every question. So the expectation is usually that good forecasters will achieve Brier scores below that. At the extreme, if you can always answer the forecasting questions correctly (“What will the result of pressing 2+2= on this calculator be?”) your score will be 0, the best possible. Conversely, if every question is a legitimate coin flip, the best possible value is in fact 0.25, anything below that is just luck.

This makes it hard to evaluate claims like “recreational markets do worse than serious markets” or “Polymarket is better at predictions than Kalshi”. On a fundamental level, recreational questions are not the same as serious questions. Similarly, the markets on Polymarket are not the same as the ones on Kalshi. And even if you pin questions that are identical across the two platforms, the fact that arbitrageurs will exploit any price differences makes the comparison all but meaningless. Any persisting discrepancies could be attributed to factors like different fee structures rather than a fundamental difference in forecasting ability.

Another important baseline to keep in mind when evaluating forecasts is their base rate. Take for example the Kalshi series on MLB games going to extra innings, which runs daily. During cycles from May 2026 to August 2026, markets resolved Yes 8.4% of the time. If you simply predict No will happen with certainty on every market, you will be right 91.6% of the time. In that case, your Brier score would be 0.0847, which naively looks pretty good. As it turns out, the best possible dumb strategy is to bet the base rate8, e.g., bet every market in that MLB series will resolve to Yes with probability 8.4%. If you do that, the Brier score will be slightly better, 0.077.

That number, the Brier score of the constant base-rate forecast, is called the uncertainty of the problem. Uncertainty is a measure of how hard the problem is. A coin flip has maximum uncertainty, 0.25. Uncertainty in the “2+2=?” task is 0. If you are using the Brier score to measure skill in a bunch of forecasting tasks, you should measure the base rate and use it to take the uncertainty into account. A given Brier score will be more impressive if uncertainty turns out to be higher.

But this already places the criticism of Kalshi’s results by @DeepDishEnjoyer into perspective. The abstract of the paper mentions that accuracy rises from 88.3% (88.4% in my replication) at a 3-month horizon to 97.2% (97.3%) at close. In the screenshot shared on X, Fable built a sample of 20,000 recent settled markets and computed a Yes rate of 5.5%. Just as Fable argued there, if that was the base rate, getting 94.5% accuracy would be trivial. All you would have to do is predict a probability of 5.5% of Yes for every question.

But that base rate is wrong. If you use a sample comparable to the paper’s, which included all resolved markets from launch through mid-2026 (I used July 21 as the cutoff for most of my replication), you don't find 5.5% of markets resolving Yes. For that sample, the base rate is actually 39.4%. Using it as a constant forecast, the accuracy you would get is 61%, which is clearly worse than the 88+% achieved by the markets. There is a catch though.

When you read these numbers naively, it looks like the markets start out pretty accurate 3 months out, at 88.4%, then keep improving until they get to 97.3% at close. There is an oddity that cuts against this interpretation however: market accuracy 1 day before resolution is only 81%. It is as if markets start out pretty accurate, then get worse for some reason until at close they finally improve becoming better than ever.

The issue is that the samples are significantly different. Notice first that market lifetime is not homogeneous. Break the entire frame into two groups, one containing only crypto+sports+intraday markets, and the other containing the rest. Here is what the distribution looks like for each group:

As you can see, noisy markets typically get resolved much sooner after being created. By construction, the sample only includes settled markets. As a consequence, the “at close” statistics include every market in the sample since they all have been settled. But only 0.6% of markets get included in the “3 months out” statistics since the majority of markets don’t live that long.

Of those markets that did exist 3 months before close, only 38% are of the noisy variety. Remember how comparing forecasting performance by checking the Brier score on different question populations is not that meaningful? As it turns out, the population of markets that exists at different time horizons is quite distinct, so it is not meaningful to compare markets at different horizons like this.

In fact, 46% of all markets are intraday. So if you compare statistics at close with statistics 1 day before resolution, you would still be looking at two quite different populations.

The authors solve this by excluding noisy markets but also by fixing a cohort of markets that existed 1 month out. Here is what the accuracy over time looks like for those:

And this is the same chart except showing the Brier scores:

Brier scores already start quite low one month before close, but they keep decreasing as the weeks go by and even during the day just before close. It’s hard to evaluate their performance without having a model of how hard the questions are, but the markets do appear to be functioning well looking at these charts.

There is another factor that could inflate the Brier score for all markets at close. In many markets trading closes “too late”: possibly after a sporting event finished. The authors do take this into account, checking what performance looks like if they re-anchor times, and it moves in inconsistent directions for sports and elections. The paper does not come up with any explanation for the discrepancy. I did not replicate this result and do not have any theories either.

Is that really calibration?

Forecasts are calibrated if their predicted probabilities match observed frequencies. For example, if you predict 100 different questions will settle to Yes each with probability 70% and then 70 of them in fact do, you are perfectly calibrated. Calibration is clearly desirable. If forecasts are not calibrated, you cannot interpret them as probabilities.

Now consider a miscalibration scenario. Suppose you predict a 70% chance for 100 events, then 99 of those happen. Then you predict a 70% chance for a different set of 100 events and 93 of them happen. Let’s assume this remains consistent: whenever you say 70%, we observe the events around 95% of the time. In this scenario your forecasts are obviously very useful and predictive. Yes, they may be miscalibrated. Your 70% really means 95%. But that is easily fixable. I can easily adjust to your miscalibration, and use your forecasts to inform my decisions.

Here is a chart of the calibration for the sample consisting of all markets at 1 month before resolution:

In a chart like this, the diagonal line represents perfect calibration. The fact that the lines are below it means the markets are a bit underconfident (edit: no, it does not). As you can see, my results match the authors’ essentially exactly. The authors include many of these calibration charts, all of which also replicate just fine. Their markets are well-calibrated! To go to the most adversarial example, take a look at the calibration for intraday markets only:

Even for these especially noisy markets, calibration 1 hour before close is fairly good. At close it gets worse, which is a bit awkward for my thesis here. I looked a bit into why that is the case but I do not have a good theory about what is going on there. I will share what I know though.

Badly miscalibrated markets there are all multi-option numeric ranges, like this market where you can bet if BTC will be above many different cutoffs on October 1st. Usually only two of the markets in a range are actually relevant at close. For example, if BTC is sitting at $110,248, the markets saying it will be above $110,200 and $110,250 still carry a lot of uncertainty. All the other markets will be close to 1¢ or 99¢. And that is it, that is what I know. Opus did spin up a story about takers being biased towards Yes but it sounded just-so and I couldn’t verify it.

In any case, the same applies for Crypto markets whose lifetime was at least a week. They are not hugging the perfect calibration diagonal, but they do just fine:

So that is the point. Kalshi’s markets are well-calibrated. Yes, the calibration diagrams look more impressive if you remove the bulk of the platform. That is to be expected though, predicting noise is a harder problem. With that said, even those noisy markets still appear reasonably calibrated.

However, there is a problem in some of their discussions. The paper keeps equivocating between calibration and the Brier score.

For example, section 4.1.1 starts by saying

Brier scores summarize calibration in a single number, whereas reliability diagrams reveal its shape.

The first part of that sentence is not true.

For example, suppose you are forecasting the result of 100 coin flips and you predict each will come up Heads with probability 50%. In this case, we know from construction you are perfectly calibrated: the coins in fact should come up Heads about half the time. If we run the experiment many times, we expect to observe pretty good calibration empirically, but it may not be perfect every time. Almost 3% of the samples will have 60 or more Heads. If that was the sample we got, you would look miscalibrated, and we can say exactly by how much. However, your Brier score will always be 0.25 regardless of the results of the coin flips. Your measured calibration can vary even if your Brier score does not.

I can’t come up with a good reason why they got this wrong. The paper even cites Murphy (1973), which presents a decomposition of the Brier score as uncertainty - resolution + (mis)calibration. I have discussed uncertainty and calibration, now I will turn to resolution. Notably, Kalshi’s paper does not share any resolution results, so a cynical reader could reasonably expect their resolution to be bad. That is not the case though.

Before I show the resolution chart, let me just explain what resolution even is. Basically resolution measures how much of the variance in the outcomes your forecasts explain. Uncertainty is the total variance in the outcomes, so resolution can never exceed uncertainty. Remember, uncertainty comes from the base rate and can be at most 0.25. Since good forecasters are usually well-calibrated, their Brier scores will typically be very close to the uncertainty of the problems minus their resolution.

A consistently miscalibrated forecaster with high resolution can still be quite useful, you could still infer informative probabilities from their forecasts. However, a perfectly calibrated forecaster with 0 resolution brings us nothing. That performance can be achieved, for example, by always betting the base rate on every question. Low-resolution forecasts do not tell us what is more versus less likely to happen.

The chart below shows resolution on the sample including only markets that existed 1 month before resolution, excluding Sports, Mentions and intraday markets:

Notice first how the Brier score is in fact very close to uncertainty minus resolution (black bar stacked on purple bar equals gray bar). That is because the markets are calibrated. But they also do display some level of resolution.

The fact that we have non-negligible resolution in highly-calibrated markets strongly suggests there is forecasting skill involved. It could still be the case that prediction markets are not doing a particularly good job compared with other baselines. It could be the case that achieving this level of performance is actually very easy. In a vacuum, it is hard to say.

People have compared the performance of markets with other forecasts, I have also done that and will write more about it in the future. Comparisons like that can help us understand just how good the markets are. If they consistently underperformed against reasonable baselines, we could argue they do not forecast well. Of course, to identify mispricings is to fix them. The markets do incentivize information to reveal itself.

To tell the same story resolution told us but using a different metric, if we use the standard Brier skill score, comparing each cohort with the base rate for its markets only, the markets still look good:

A Brier skill score of 1 means perfect skill, 0 means performance as good as the base rate. Clearly, there is some amount of skill being displayed. Notice that the strongly recreational crypto+sports+intraday markets display less skill than the other markets, as expected, but still display positive skill.

Inconsistent numbers and puzzling results

There are a few inconsistencies in the paper’s numbers:

• The abstract says Kalshi comprises 2,243,741 markets from its launch in 2021 through mid-2026, but summing the per-category counts in table 1 yields a larger number, 2,324,075.

• Table 1 says there were 6,064 election markets in the sample, table 4 has 4,226.

• Summing the bucket counts at close in table 2 yields 79,504 markets, table 3 gives 85,315.

I don’t think these mean much. They likely arise from slight differences in how the samples are built. Those differences either went over my head or were not properly explained, which happens. Though it should be noted that both GPT-6 and Fable 5.1 consistently caught those apparent inconsistencies when prompted to look for mistakes in the paper.

Fable’s analysis screenshotted in the X post by @DeepDishEnjoyer brings up 8 “other crimes” by the paper. Some of those I have echoed but most I just consider overblown. I will address one of them here.

Fable claims that the authors excluding markets priced below 2¢ or above 98¢ is fishy since Bürgi et al. (2025) found a favorite-longshot bias. I think the criticism is misguided, but while investigating it I realized that calibration estimates do rely on binning choices, and the calibration charts in the paper are coarse enough that I had no idea how tail markets perform. So I charted calibration for two tail ranges: 0-5% and 95-100%, with small bins. The charts look quite bad (edit: it isn't really bad9). Take a look first at the one for the full sample of all settled markets ever:

The 95-100% tail improves a bit if we throw away sports, mentions and intraday:

Kalshi’s API does not allow me to track traders’ identities, which is fair but means that I can’t replicate the association between trader count and Brier score. They give us this chart:

There is an association between good performance (low Brier) and number of traders, but it is hard to confidently ascertain why. Maybe easy markets attract more traders. Maybe a high number of noise traders attracts sharks. The authors do draw inferences of the sort “adding more traders helps but waiting a bit often helps more”, which is tempting but I think does not work? Everything is just too confounded.

I was able to investigate the relation between trading volume and the Brier score, but there is enough I want to say about it that I have decided to leave that for a later article.

In sum, though arguably too much of Kalshi is pure gambling, their markets are about as good as the paper they released claims they are. Their results are unsurprising and reasonably accurate. They get some things wrong and do not frame results in a way that goes strongly against Kalshi’s interests, but I don’t see why we should expect that. I have no idea where the $40B valuation in Kalshi’s current investment round comes from, but the picture in my mind is that they offer almost zero economic value if you take gambling away. Personally what made me interested in prediction markets was the potential for applications in business and the nonprofit world, so this is disappointing. Still builders interested in using prediction markets as a primitive for decision-making might gain something from paying attention to how Kalshi’s markets work. I may also try to explicitly learn such lessons in a future post.

Subscribe now

In the last two years, I have read 10 Chinese visual novels1 spanning 400 hours2 and over 3 million characters3 worth of content. For comparison, The Three-Body Problem has just over 200k characters, so I read 15 times the equivalent of that novel, which improved my literacy quite a lot. Additionally, it was at least as fun as any reasonable option to build my skills up from an early intermediate level.

I’ll share a bit about the experience below, and give tips for those seeking to learn Mandarin using VNs.

Visual novels for language learning

The Heavenly Path does recommend some visual novels, including a few I haven’t played yet. However, I first got the idea of using them as a learning device from the Deep Weeb podcast, which targets Japanese learners. Visual novels as a genre were invented in Japan, and most titles are Japanese. Japanese learner “the Doth” achieved notoriety for the remarkable feat of passing the Japanese N1 exam with less than 15 months of study, and he did make extensive use of visual novels to practice reading. His and other similar stories covered by Deep Weeb made an impression on me.

As an aside, beyond just making me aware of visual novels, I’ve found guides for Japanese learners to be quite informative for other reasons. As awesome as the Heavenly Path is, I still got new ideas from TheMoeWay, and part of what I’m trying to do here is help bridge the gap.

The benefits of visual novels

There is a smooth progression of Chinese graded readers one can follow4, but the transition from those to books written for natives is harsh. There comes a point where even children’s books involve an inordinate amount of dictionary look-ups. Visual novels shine at precisely this level, helping us break the barrier from learners’ materials to native media. Several factors contribute to this.

For one, though the language used can be complicated for some titles, in my experience it is typically much easier than what one finds in books. That’s because the stories tend to be aimed at young audiences and are almost always dialogue-driven. Moreover, when playing a VN, instead of facing a huge scary wall of Chinese text as one would in a book, there is only one tiny paragraph to be read at any given point. You read it quickly, press space, read the next one, press space again, rinse and repeat. This whole cycle has a mild dopaminergic effect, gobbling sentences up then pressing space is kind of addictive.

The audiovisual elements also contribute a lot make a boring story palatable. The usual path towards literacy in a foreign language involves some amount of putting up with uninteresting stuff because it is more suitable to one’s level than other options. The stories in visual novels are not always the best, but the anime art, music and voice acting end up making the experience enjoyable. I remember Gaokao.Love.100Days came as a breath of fresh air. When I read it, I wasn’t ready to upgrade to actual novels, but at the same time couldn’t stomach reading more graded readers or children’s books. Just one minor caveat though: the audiovisuals are a two-edged sword, as some learners might be repealed by their silliness and inclusion of fan service5.

The dialogical nature of the stories plus the animations and sounds make reading visual novels for long stretches of time less fatiguing, so we end up reading more. Those games are usually very long as well, which means more opportunity for practice. It is not unusual for them to take dozens of hours to be finished by natives, with learners requiring several times as long. Reading the same title for a long time helps us get familiar with its style, build confidence and increase speed. The stories are commonly choose-your-own-adventure style, which adds some replayability as we get to explore different story routes. The fact that early intermediate learners struggle to find suitable reading materials further compounds the value of length within any given title6.

What the setup looks like

It’s hard to configure all the software most learners use to play VNs working. When the setup runs smoothly, the experience is surprisingly ergonomic, but it is an absolute kludge for sure. I’ll refer the reader to other guides with detailed instructions, and just explain the mechanics in simple terms here to clarify what is and isn’t possible at the time of writing.

We play the game on Windows desktop, with a text hooker7 attached to it. This text hooker is the key piece of software which makes easy dictionary look-ups possible. Essentially, what it does is to grab the game’s dialogue lines from memory and send them to the clipboard. Once the text is there, other software can be used to render it on a special webpage. Off-the-shelf browser extensions8 can then be used on that page to look the definition of a word up by just moving the mouse over it.

With a little more effort9, one can get to a point where creating multimedia Anki flashcards, complete with a screenshot and audio recording, takes 10-20 seconds.

What this all amounts to is making it possible to play desktop games with instantaneous dictionary look-ups and easy flashcard creation. Now, all the magic in the setup relies on the text hooker being able to do its job, which it can’t always do. Many games don’t play well with hookers, and even for those which do, some experimentation with settings may be needed. The games mentioned in this article can be configured to work unless otherwise specified.

The alternative to using text hookers is relying on OCR, which is what people generally do to read manhua. Early intermediate learners who care more about their input being comprehensible than about playing any one specific title would do well to prioritize games which work with the hooker. Free OCR software is not that reliable, and at the early intermediate stage look-ups are frequent. I am currently at a level where I do feel tempted to go back and try out some titles which I passed earlier due to lack of hooker support, like WILL: A Wonderful World (VNDB; Steam).

As one final tip before I move on to discussing a specific game, when searching for information on titles, the Visual Novel Database (VNDB) is invaluable. It includes user reviews, tags describing the game content, and many other filters, along with powerful search features. For example, this page lists the most recent releases whose primary language is “Chinese (simplified)”, tagged by users as having no sexual content and whose rating on the website is at least 4.

Gaokao.Love.100Days

I originally wanted to go over all the games I played and share some tips on how to select which game to play, but ultimately decided to leave that for another day and just discuss my experience playing Gaokao.Love.100Days here, which is as good of an option as any for someone new to learning Mandarin by reading VNs.

Gaokao was the first visual novels I ever read, and its premise immediately hooked me. We will sit the life-defining gaokao in 100 days, having just started a relationship with the cute classmate we had a crush on. Because teachers and parents frown upon relationships, we have to keep it a secret. During the next few months, we need to juggle parental pressure, working hard to increase our score, and dating, which both ourselves and the girl, Muxin, are clueless about.

Since this title was my introduction to the genre, I picked up many tropes and conventions from it. To start with, notice how I described the story in first-person. Visual novels usually are presented from the perspective of the protagonist, who is never shown on screen. The most important characters have both voice acting and sprites, secondary characters can have sprites but no voice acting, and minor ones sometimes don’t even get a name. I’m speaking in generalizations, of course, there are exceptions for everything and different subgenres may go with different conventions.

In Gaokao, there are several lengthy dialogues between, e.g., the protagonist and his parents, all of whom have no sprites or voice acting. I found scenes like those, with only text and no audiovisual components, to be boring in almost all of the visual novels I played. Mostly that is because those missing elements help keep my attention when they are present, but another likely reason is that the scenes including them received more care overall. The creators thought it was worth spending their limited resources to add some eye (and ear) candy, which implies they were better to begin with.

Another issue is that Gaokao has dozens of different endings. The way this works is that as the game progresses, you make choices. Sometimes in the middle of a dialogue you have to choose between 2 or 3 options for how to respond to a situation. But more frequently, at the end of each of the 100 days of the game, you get faced with the screen below, in which you choose which subject to study, or whether to rest, talk to the girlfriend, etc.

Anyway, depending on the choices you make, you unlock different story routes. These routes will focus more on different characters, and each of them will include a few possible endings. Any given route can contain a good, a neutral and a bad ending. You can always save just before making a choice, proceed to see how the story ends, then load back, make a different choice, and see the next ending. Doing this, you could attempt to collect all endings, which is one of the game’s achievements. To make this less repetitive, there is a skip function, which will quickly skip over previously seen lines, stopping only upon reaching a new scene.

Originally, I had received the advice to always have a guide open and to try to get to a good ending, because it would be more fun. For some games, this matters less, but in the case of Gaokao, the choices I made led me to the “no route” story route, so the latter half of my first playthrough was super boring. It consisted of me going through the motions, clicking to study Math or Chinese, with little actual story to keep my interest. I advise the reader to not be afraid of spoilers, and to do the right thing and follow a guide. In this particular game, a good story route is way more fun than a bad one.

In the end, even with the skip function, after a few playthroughs the novelty wore off and I was bored of this game, so I didn’t get even close to seeing all endings. I think I went through 3-5 of the major story routes though, and one thing I liked about doing that is how they complemented each other. A character does something in one branch, but their motivation for that is explained in the other. There is some lore build-up as well, different routes fill in on the characters’ backgrounds and the school’s history, and there are also some wacky easter eggs to be found.

My Steam account has ~70 hours of play time recorded for Gaokao.Love.100Days. All in all, I consider it a great title and would strongly recommend it for any learner who feels some amount of curiosity and who is willing to put up with the pain of setting up the text hooker and everything else. This game most definitely wouldn’t appeal to me if it wasn’t for the added benefit of language learning, but that is irrelevant. I would only rarely read any long story at all if I wasn’t gaining anything besides pure enjoyment, and I see the possibility of being exposed to new types of media, which can embed cultural differences, as one of the benefits of learning a foreign language as a hobby.

Subscribe now

Before I delve into where I am at in specific subskills, let me try to give an overall impression of my level. I don’t know the exact amount since I haven’t really tracked, but I have spent a few thousand hours studying Mandarin, the majority of which was reading or listening to content made for natives. Recently, I took a practice HSK 6 exam to get an idea of where I am at, and got a marginally passing score. In the HSK 2.0, which is the one I used, this level is the highest, and the HSK 6 is sometimes compared to a B2/C1-ish level in the CEFR. With that said, I think I am not there yet, specially because I can’t express myself well enough — the HSK is very focused on reading and listening, which is what I practiced the most. If someone asks me what my level is, I just round it down to an indeterminate “intermediate”. My plan is to keep grinding until that label starts to feel like an understatement.

Character recognition and handwriting

In order to read Chinese, one has to become familiar with thousands of characters. Going off my Anki stats, the total number of characters I have ever studied is ~3,400, though the number I can confidently recall is likely in the 2,600-3,000 range. About a year ago, I gave up on Anki, since doing my daily reviews was stressing me out and I wasn’t getting much benefit from learning relatively rare words anyway. Right now though, I'm rebooting that habit, with my main goal being increasing the number of characters I know to 5,000 within 12 months. 5,000 characters should be enough to make dictionary look-ups much less frequent1. I expect that getting that far while also reading a lot will do wonders for my literacy.

I’m also just starting to learn how to handwrite my first characters. Traditionally, handwriting was seen as an essential, albeit awfully time-consuming, step towards literacy. Nowadays though, many students prefer to skip it and learn only the much easier skill of recognizing characters. Getting good at that is already enough to read prose written with clear computer fonts as well as to type on a computer. However, ignoring handwriting altogether like I did makes it hard for me to recognize isolated characters, or those written with different fonts. As I will explain, those are not useless skills, and those limitations carry some bite.

To start with, there are many characters that look similar, and since I can’t reliably distinguish them in isolation, I usually rely heavily on top-down processing to understand a text. For example, I am embarrassed to admit I still sometimes mix up the elementary characters 乌 and 鸟. Usually, when either occurs in a sentence, the context makes clear which of the two it is. One important exception is names, which can contain quite arbitrary combinations of characters, so if 乌 is one of the syllables in a name, I could end up misreading it. Needless to say, most texts include names of people, places, spells, rivers and whatnot, and names can also be seen in storefronts, so I am constantly being bogged down by this limitation either when reading or just looking at commercial streets in China.

As for the other problem, not understanding characters that are not written in neat computer fonts, I think it could be circumvented if I restricted myself to relatively easy ones. For instance, the screenshot below is from See You Again season 4, and the text is written in a computer font that emulates handwriting. I get confused about a few of the characters, but I think with enough practice I could learn to read something like this very comfortably.

However, when it comes to actual handwriting, natives can merge or omit strokes in ways that are difficult to parse if one is not familiar with how characters are written2. Whenever something the participants wrote was shown on camera in See You Again, I simply wouldn’t get it. Also, the very fact that viewers of the program were expected to be able to read it is proof that this is not an useless ability.

My plan is to learn to handwrite 1,000-1,500 characters. That is way less than the ~5,000 I expect to be able to recognize a year from now, but I think it will be enough for my purposes. The hope is that by internalizing stroke order rules and most radicals, I will develop a closer relationship with the components that make up characters in general. This in turn will lead me to have a much clearer mental picture of even characters that I can only recognize and not write. But frankly, even if that turns out to not be the case, there is no way that learning to write 5,000+ characters is worth my time. I will derive whatever benefits I can from the characters I end up learning to write by hand, and then move on.

Catching-up with listening; starting to speak

Roughly 3 months ago I started practicing speaking, following my first period of solid progress in listening. Before, the fact that my listening sucked discouraged me from wasting time speaking. My reasoning was that listening within a conversation is strictly harder than listening to a podcast, because in addition to the need to understand what is being said, I have to pay attention to social dynamics and come up with what I will say next. That reasoning was always flawed since in one-on-one conversations people can slow down to make it easier for me to understand what they are saying. Still, I wasn’t terribly excited about having simplistic conversations in slow motion, and I didn’t mind waiting a bit.

Anyway, at some point I watched a video in Cantonese, which I shouldn’t understand at all, and noticed that I was able to follow along close to 100% by reading the subtitles in Mandarin. This sucked because the bulk of my listening practice at the time consisted precisely of watching videos while reading subtitles. In the year prior, as I explored different content, I went for harder and harder videos since they tended to be more interesting. Without realizing, I was relying more and more on the subs. When I finally noticed that, some testing made it clear that my listening was fairly bad, so I had to focus more on it.

The first thing I did was to stop reading the subtitles. This simple change then led me to chase easier content, until the point where I was watching channels targeting learners instead of natives, and those were still challenging. The two I enjoy the most are Dashu Mandarin and Chinese Podcast with Shenglan. I found the latter so interesting that I speedran all of Shenglan’s videos on YouTube, then all 300+ premium podcast episodes on her Patreon, within a few months. Way before I was done with all of that though, my listening had already improved substantially, and I thought it was time to finally start practicing speaking for the first time ever.

Having listened to so many episodes of Shenglan’s podcast, I decided to attend her classes for a couple of months. The decision to start with her as a tutor was great, she’s super cool to talk to, extremely patient. Speaking for the first time after studying the language for so long was daunting, and having someone whose content I enjoyed as my conversational partner smoothed out this transition. It also built my confidence, I’m now confident that I can speak, even if it’s guaranteed to have many mistakes and awkward pauses. That’s still enough to have exchanges that both parties find engaging and, with some patience, talk about relatively complex subjects.

Nowadays, I routinely listen to podcasts made for natives, though I wouldn't be at 100% of comprehension even if I sat down and paid attention, let alone while multitasking. Also, they are all handpicked for being easy. The videos I enjoy watching the most still require lots of pausing and looking up words. This is still the best my listening has ever been, and I'm proud I got this far.

Starting to read traditional; continuing simplified

When it comes to reading in simplified, it’s still arguably my best ability. I’ve read about 10 visual novels and 6 books, not counting children’s books or graded readers. Currently, I’m reading my first ever xianxia3, Cultivation Chat Group, which the Heavenly Path guide recommends as a good entry-point into the genre. Given its length, odds are I won’t go all the way to the end, but so far the story is lots of fun. Earlier this year I read a few short stories by Liu Cixin and enjoyed those as well.

Late last year, I finally started reading traditional. The first thing I read was the Taiwanese visual novel The Rainy Port Keelung. I went in head first without any preparations, just armed with a dictionary. Even the most basic characters that have only minor changes from simplified to traditional required a look-up, and then it took me an absurd amount of look-ups to absorb those characters. It was quite tedious. In the beginning, I was reading a mere 4,000-5,000 characters per hour, in line with where I was when I read my first Chinese visual novel ever, Gaokao.Love.100Days. When I started with The Rainy Port Keelung though, I was used to reading at 2-3x that speed, so starting traditional felt like becoming a newbie again.

It took a long time to start to feel comfortable, and even now it isn’t anything like reading simplified. Fortunately however, after about 70 hours of practice, I am sometimes reading more than 12,000 characters per hour now. I also text with Taiwanese people all the time on HelloTalk with no issues, they write in traditional, I reply in simplified. At least when it comes to easier characters, I rarely misread them or need the dictionary. I’ll still stick to visual novels for a while before tackling easy books, but I’m glad that I didn’t feel the need to go as far back as reading graded readers again to pick up traditional.

The challenges of learning Chinese at this level

People sometimes say that having conversations in Chinese is easy and only the written language is hard, supposedly because of the characters. That could well be the case at an elementary level, after all, a native can always slow down and use simple words to make a conversation possible, but getting over the hurdle of memorizing the first couple thousand characters takes a lot of time, not to speak of other complications like lack of spaces between words. But I’d say that at the intermediate level, listening is trickier than reading. When natives are expressing themselves without taking pains to go slow and simplify, they can speak quite fast, with tons of regional accents and complex vocabulary. By contrast, while reading, I go at my own pace and have convenient access to the dictionary.

My guess is that the situation will reverse again at the advanced level, and reading will become more challenging. That’s because I expect the oral language to be comparatively capped in its complexity, and written texts to go much deeper. Also, there is the issue of skimming ability and reading speed. Having grown up with the Latin alphabet, I’m not convinced I’ll ever be able to get my reading to a level where it feels as effortless as reading in Portuguese or English.

That is due to the script, but some aspects of the language would be tricky even if China somehow made it work with an alphabet. Chengyu comes to mind. Yes, one can get away with just treating them like regular vocabulary words and learn the ones that are actually used frequently through sheer exposure, but I suspect that at the really advanced levels of literacy, some dedicated effort will pay off. Another issue is Classical Chinese4. The way I understand it, that ancient language itself is not really used in modern media, but there are some classically-influenced usages that may become more transparent by spending some time to pick up the basics. I would also say that more broadly, familiarity with the culture is a big thing. China really has a long and fascinating history, and even if it that somehow wasn’t relevant, it is so huge that there would still unavoidably be a lot to learn about just to get up-to-speed with contemporary culture. Chinese internet memes deserve their own dedicated bookshelf.

Why I got on this path and how things will unfold

Overall, my Chinese is moving along at a reasonable pace. Most days, I spend 2 hours on it, and I’ve been doing so for a little under 2 years. Before then I had already read a few children’s books and some 50 or so graded readers, and had about 3,000 words under my belt on Anki, so I absolutely did not start from scratch. With that said, until that point, I never really prioritized Chinese and progressed only very slowly. I decided to crank up my effort because when living in the countryside close to the end of my masters I wanted an escapist hobby that I could do at home. Reading Chinese visual novels and watching donghua perfectly fit the bill. I was stressed and bored, and consuming tons of Chinese entertainment media was a productive way to get through that.

The most unexpected thing I found along the way is how intriguing modern Chinese society turned out to be. The more I learn, the more fascinated I get about this culture. Shenglan’s podcast has been a treasure trove in this regard, introducing me to tons of memes and little corners of life one wouldn’t even think about otherwise without stepping foot on China.

I’ve been doing this as a hobby and mostly disconnected from speakers, and I’m curious how things will unfold as I integrate more and more into the Chinese-speaking world. Acquiring English changed my life, and I expect Mandarin to have a comparable effect, especially as I reach the upper-intermediate level and beyond.

Subscribe now