
Every so often you read a paper that quietly rearranges an assumption you did
not know you were carrying. For me, the Lottery Ticket Hypothesis is one of
those. It is not new — Frankle and Carbin published it at ICLR in 2019 — but it
keeps coming back to mind whenever someone asks why their model has to be so
large.

Here is the thing that never sat right with me. You train a big network, you
prune 90% of its weights, and the pruned model does fine. Faster, smaller,
cheaper. So why not just build the small one to begin with? The standard answer
was that it doesn't work: train a small network from scratch and it learns
badly. Everyone accepted that and moved on. Big models, the reasoning went, are
simply what learning requires.

Frankle and Carbin took the obvious follow-up question seriously and found
something stranger.

## The part that surprised me

Their claim is that a randomly initialized network already contains a small
subnetwork — the winning ticket — that, trained on its own, matches the full
model. The catch is in the word "already." The ticket is not something training
creates. It is there from initialization, decided by the dice roll of the random
weights, and training merely finds it.

The procedure that gets you there is almost annoyingly simple. Train the big
network. Prune the smallest weights. Then take what's left and reset those
weights to the exact values they had before training ever started, and train
again.

That last step is where it stops being a pruning trick. Reset the surviving
weights to _new_ random values instead of the original ones — same architecture,
same sparsity, same everything but the specific numbers — and it falls apart.
Slower to learn, worse accuracy. The structure alone isn't enough. Neither are
lucky numbers alone. What works is the pairing: those particular weights sitting
in those particular positions.

I find that genuinely odd, in a good way. It means the thing doing the work was
never the 100 million parameters. It was a handful of them that happened to
start in a good place, and the other 99 million were the price of the search.

## What the numbers look like

The results are consistent enough to be uncomfortable. Across the models they
tested, tickets land somewhere around 10-20% of the original size, reach the
target accuracy two to three times faster, and often generalize slightly
_better_ than the network they came out of. Some go much further — down to a few
percent of the original parameter count and still training fine.

Large models are less cooperative. At standard learning rates the method stops
finding tickets; drop the learning rate, or warm it up gradually, and they
reappear. Which is its own small hint that whatever makes a ticket work is
fragile early in training, and easy to stomp on if you move too fast.

## Why it reframes the question

The framing I keep returning to is that scale buys you lottery tickets. You are
not training a big model because every parameter earns its keep. You are
training it because more parameters means more independent draws, and more draws
means a better chance that one of them started somewhere trainable.
Overparameterization isn't capacity. It's search.

That reading also explains the generalization result without much hand-waving.
The ticket is smaller than the network that contained it and does better on
unseen data, which is roughly what you'd expect if the extra parameters were
scaffolding rather than substance.

There is an echo of this in how frontier models are built now.
Mixture-of-Experts architectures route each token through a small slice of a
very large parameter count — Korea's recent models activate somewhere between 6%
and 10% of their total weights at a time. The mechanism is different: MoE picks
a different subset per input, while a winning ticket is one fixed subset for
everything. But the underlying bet is the same one, that the useful computation
is sparse and the bulk exists to make finding it possible.

## The catch, and why it still matters in practice

The honest limitation is that you have to train the full model to find the
ticket, which means the hypothesis does not — yet — make training cheaper. It
makes _inference_ cheaper, which for anything running in production is usually
where the money actually goes. Train once, prune hard, deploy the ticket, keep
the accuracy.

The open question, the one that would change everything if someone answers it,
is whether tickets can be identified before training rather than after. If you
could read the initialization and know where the winner is, you would skip the
expensive part entirely. Nobody has done that convincingly. A lot of people are
trying.

Until then, I think the value of this paper is mostly in how it makes you think
about your own models. The next time one feels too big, it is worth sitting with
the possibility that it isn't big because the problem is hard. It's big because
you're still buying tickets.

---

The paper is
[The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks](https://arxiv.org/abs/1803.03635)
by Jonathan Frankle and Michael Carbin (MIT CSAIL, ICLR 2019). It is unusually
readable for a research paper and the experiments are laid out clearly enough to
reproduce. Worth an hour of your evening.


Optimize Your Models

A short note on a paper that has stuck with me: most of a trained neural network can be thrown away, and the small thing left inside was never the point of the big model in the first place.

The Lottery Ticket Hypothesis: Sparse Networks That Actually Train

lottery-ticket-hypothesis-neural-network-pruning


## Codex Is Now Open Source

OpenAI just made a significant move: **Codex, their AI coding assistant CLI, now
works with open-source models through Ollama.**

This means you can run a powerful code generation tool locally without relying
on OpenAI's proprietary API. The CLI can read, modify, and execute code in your
working directory using models like `gpt-oss:20b` or `gpt-oss:120b`.

## Getting Started

Setup is straightforward:

```bash
npm install -g @openai/codex
codex --oss
```

By default, it uses the local `gpt-oss:20b` model. For larger tasks, switch to a
bigger model:

```bash
codex --oss -m gpt-oss:120b
```

**Important:** Codex requires a large context window. Ollama's default is only
4096 tokens, but coding tools need at least 32K. Set this before running:

```bash
OLLAMA_CONTEXT_LENGTH=32000 ollama serve
```

Keep in mind that larger context windows require more VRAM. Verify your setup
with `ollama ps` to confirm the context size is allocated correctly.

## Cloud Options

If local hardware isn't sufficient, Ollama Cloud models are fully supported:

```bash
codex --oss -m gpt-oss:120b-cloud
```

This gives you the flexibility to run smaller models locally for quick tasks
while offloading heavier workloads to the cloud.

## Why This Matters

Three reasons this is significant:

**1. Local-first AI development** - Run code generation entirely on your
machine. No API keys, no usage limits, no data leaving your system.

**2. Model flexibility** - Choose the model that fits your hardware and task.
The 20B parameter model works for most coding tasks; scale up to 120B when
needed.

**3. Lower barrier to entry** - Open-source models and local execution
democratize access to AI coding tools. You're no longer locked into a
subscription or pay-per-token model.

## What This Signals

OpenAI open-sourcing Codex reflects a broader industry trend. As open-weight
models become increasingly capable, proprietary APIs face pressure to offer more
value or open up.

For developers, this is a win. More options, more control, and the ability to
integrate AI coding assistance into workflows without vendor lock-in.

## The Bottom Line

Codex with Ollama is worth trying if you're building software. Local execution,
open models, and a simple CLI make it easy to experiment without commitment.

The gap between proprietary and open-source AI tools continues to narrow—and
developers benefit every time it does.


Build with AI

OpenAI has open-sourced Codex, their AI-powered coding assistant CLI. Developers can now run it locally with open-weight models through Ollama, making powerful code generation accessible to everyone.

OpenAI Codex Goes Open Source with Ollama Integration

openai-codex-open-source-ollama


Four frontier models landed in Korea in the last week of December. A.X K1 from
an SK Telecom consortium on the 27th, VAETKI and K-EXAONE on the 30th,
Solar-Open on the 31st. Announced days apart, most of them presented at the same
government event at COEX in Seoul, which tells you this was scheduled rather
than coincidental.

The headline numbers are big — 519 billion parameters for A.X K1, 236 billion
for LG's K-EXAONE — and I think the headline numbers are the least interesting
part. What caught my attention is that all four made the same two bets, and
neither bet is the one the US and Chinese labs have been making.

## Almost nothing is switched on

Every one of these models is sparse. Look at what actually runs per token:

A.X K1 activates 33B of its 519B. VAETKI activates 10-11B of 112B. Solar-Open,
12B of 102B. K-EXAONE, 23B of 236B. That's between 6% and 12% of the declared
parameter count doing any work at a given moment. The rest is Mixture of Experts
scaffolding — capacity that exists so the router has somewhere to route, not
because it all fires.

This is the number that matters commercially, because inference cost tracks
active parameters, not total ones. A 236B model that behaves like a 23B model at
serving time has completely different economics from a 236B dense model, and LG
leaned into that: K-EXAONE runs on A100s. Not H100s, not B200s. Hardware you can
already rent cheaply, or already own.

There's a pattern here that shows up elsewhere too. The Lottery Ticket
Hypothesis found that a trained network's useful computation lives in maybe
10-20% of its weights, with the rest serving as search space. MoE arrives at a
similar ratio from the opposite direction — decide up front that most of the
model stays dark, and route around it. Different mechanism, same underlying
suspicion: the parameters that matter are a small fraction of the parameters you
paid for.

## Open, and openly strategic

The second bet is that all four are open or heading there. Solar-Open shipped
under an Apache-2.0-style license with weights on Hugging Face the day it was
announced. K-EXAONE is up as `LGAI-EXAONE/K-EXAONE-236B-A23B`. VAETKI is
released. A.X K1 has a stated plan for open weights, APIs, and partial training
data disclosure.

I don't read this as generosity. Korea is fourth or fifth in line for attention
in a race that gets covered as US-versus-China, and open weights are the
cheapest way to get built on. If developers fine-tune your model, you become
infrastructure without having to win a marketing war. SK Telecom is explicit
about the ambition — it's positioning A.X K1 as a teacher model for distilling
smaller ones, with 20-plus institutions signed up.

The distribution story is the part that's easy to miss from outside. A.X K1 has
SK Telecom's A-DoT platform behind it and something like 20 million users of
consumer surface area. Most open model releases are weights and a README. This
one arrives attached to a telco.

## Are they actually good

Somewhat, in the places they're aiming at.

LG reports K-EXAONE averaging 72.03 against Qwen3's 69.37 and GPT-OSS-120B's
69.79 — a real result, and also a vendor-reported one on a benchmark suite the
vendor chose. I'd hold it loosely until independent evaluations land. Solar-Open
trained on roughly 19.7 trillion tokens with a 128K context; K-EXAONE natively
handles 256K and claims a 70% memory reduction from its 3:1 hybrid attention
scheme.

None of that makes them the best models in the world at English reasoning or
creative work, and I don't think anyone involved is claiming so. What they are
is credible at Korean and English, deployable on-premise, cheap to serve, and
licensed permissively. For an enterprise that needs a model inside its own
network for regulatory reasons, that combination beats a slightly smarter model
behind someone else's API.

## Why I think this is worth noticing

The interesting thing isn't that Korea has frontier models. It's the argument
implicit in how they built them: that the frontier worth competing on is
deployment economics, not benchmark ceilings.

Everyone hits diminishing returns on scale eventually. The labs that got there
first are now spending enormous effort on making inference cheaper —
distillation, quantization, sparsity, routing. Korea skipped to that phase,
partly by necessity. When you have less compute and less capital than your
competitors, efficiency isn't a virtue you choose, it's the only door available.

Which makes this a useful natural experiment. If a coordinated national program
with a fraction of the budget can produce open, sparse, A100-deployable models
that are competitive for most real workloads, that says something about how much
of the current spending is buying capability versus buying the last few points
on a leaderboard.

We'll know more in six months, when people have actually fine-tuned these things
and reported back. For now the thing I'd take away is narrower and more durable:
sparse and open is a coherent strategy, four serious teams converged on it in
the same week, and none of them needed the biggest cluster in the world to do
it.

---

Primary sources if you want to dig in:
[SK Telecom on A.X K1](https://www.prnewswire.com/news-releases/sk-telecom-unveils-ax-k1-koreas-first-500b-scale-hyperscale-ai-model-302649835.html),
[Solar-Open on Hugging Face](https://huggingface.co/upstage/Solar-Open-100B),
[K-EXAONE on Hugging Face](https://huggingface.co/LGAI-EXAONE/K-EXAONE-236B-A23B),
and The Korea Times on
[the national foundation model project](https://www.koreatimes.co.kr/business/tech-science/20251230/national-ai-foundation-model-project-unveils-initial-results).


Explore AI Solutions

Four Korean foundation models shipped in the last week of 2025, all of them open, all of them sparse. A note on what that combination says about where the useful part of this industry is heading.

Korea's AI Blitz: Four Frontier Models, One Week

korea-ai-revolution-2025


Nobody gets into payments because they are excited about PCI DSS. It arrives
later, usually as a spreadsheet from someone in finance, and by then the
architecture that will make compliance easy or miserable has already been
written.

That timing is most of the problem. Compliance gets treated as an audit you pass
rather than a property of the system, and the two produce very different
engineering. I want to write down how I think about it, because the framing
matters more than any particular tool.

## The scope question is the whole game

PCI DSS applies to systems that store, process, or transmit cardholder data.
Every requirement — the encryption, the access controls, the logging, the
quarterly scans — applies to whatever falls inside that boundary. So the single
highest-leverage decision available to you is making that boundary small.

Most teams discover this backwards. They build a system where card numbers flow
through their own API, land in their own database, and appear in their own logs,
and then they go looking for software to secure all of it. That is expensive
forever. The alternative is to arrange things so raw card data never touches
your servers at all: the payment provider's iframe or SDK collects it, you get
back a token, and your systems only ever handle the token.

The difference in ongoing cost is not marginal. It's the difference between a
short self-assessment questionnaire and a full audit with a QSA, every year,
plus the engineering work to keep every in-scope system compliant between
audits.

If you take one thing from this: spend your design effort on shrinking scope,
not on securing a large scope well.

## What the tooling actually buys you

Compliance software is genuinely useful, but it's worth being clear about which
problem each category solves — and which ones it doesn't.

| Capability                      | What it actually does for you                                 | What it won't do                                                              |
| ------------------------------- | ------------------------------------------------------------- | ----------------------------------------------------------------------------- |
| Tokenization / vaulting         | Removes raw PAN from your systems — the biggest scope reducer | Help if you've already logged card numbers somewhere upstream                 |
| Encryption at rest / in transit | Satisfies a requirement and limits blast radius               | Protect against an attacker with valid application credentials                |
| MFA and access control          | Cuts off the most common intrusion path into admin systems    | Cover service accounts and CI pipelines unless you deliberately scope them in |
| Continuous monitoring / IDS     | Shortens time-to-detection, which is what breach cost tracks  | Mean anything if nobody owns the alerts                                       |
| Automated evidence collection   | Turns audit prep from weeks of screenshots into a report      | Make you compliant — it documents whatever state you're actually in           |
| Vulnerability scanning          | Catches the known-CVE class of problem on a schedule          | Find logic flaws in how your own code handles cardholder data                 |

The pattern in that right-hand column is worth sitting with. Tools are good at
the parts of PCI that are mechanical and continuous — scanning, evidence,
enforcement of settings. They're bad at the parts that are architectural
decisions or organizational habits. A dashboard showing green does not tell you
that someone added a debug log line last sprint that writes the full request
body.

## Breaches are rarely exotic

The cases people cite as cautionary tales are notable for how ordinary the
failures were.

| Incident           | Scale                 | Root cause                                                              |
| ------------------ | --------------------- | ----------------------------------------------------------------------- |
| Equifax (2017)     | ~147M people affected | Known Apache Struts vulnerability, patch available, not applied in time |
| Capital One (2019) | ~100M people affected | Misconfigured web application firewall allowing access to cloud storage |

Neither was a novel attack. One was patch management, one was a cloud
configuration mistake — both squarely within what a compliance program is
supposed to catch, at organizations that certainly had compliance programs. The
lesson I draw isn't "buy better tools." It's that the boring operational
disciplines are the ones that actually fail, and no amount of tooling
compensates for nobody being on the hook for them.

Meanwhile the fraud numbers keep climbing — the Nilson Report has projected
global card fraud losses passing $35 billion a year — which is why the standard
keeps tightening and why regulators keep escalating penalties.

## What I'd actually ask a vendor

If you're evaluating compliance software, most of the demo will be dashboards.
The questions that separate the useful products from the pretty ones are
narrower:

Does it reduce my scope, or just document it? Tokenization and hosted fields
change what you're liable for; a monitoring dashboard doesn't. Can it produce
evidence continuously rather than at audit time, so that "are we compliant right
now" is answerable on a Tuesday? Does it integrate with the payment stack you
already run, or does adopting it mean re-plumbing your checkout? And when it
fires an alert, does it land somewhere a human is accountable for it, or in a
channel everyone muted?

The last one isn't a product question, strictly. But it's the one that
determines whether any of this works.

## The part that's worth caring about

Compliance and security aren't the same thing, and pretending otherwise is how
organizations end up certified and breached in the same year. PCI DSS is a floor
— a reasonable, well-considered floor, but a floor. Passing it means you haven't
done anything obviously negligent. It doesn't mean you're safe.

What it does give you, if you approach it as architecture rather than paperwork,
is a forcing function to answer questions you should be answering anyway. Where
does cardholder data live? Who can reach it? How would we know if someone did?
Teams that can answer those crisply tend to pass audits as a side effect.

---

If you're building payment infrastructure and want a second pair of eyes on the
scope boundary before it hardens, that's a conversation
[we're happy to have](/#contact).

Sources: the [Nilson Report](https://nilsonreport.com) for fraud projections
(mostly paywalled), the FTC's
[data security enforcement page](https://www.ftc.gov/news-events/topics/privacy-security/data-security),
and the DOJ's
[Equifax settlement announcement](https://www.justice.gov/opa/pr/equifax-inc-agrees-pay-575-million-settle-ftc-cfpb-and-state-attorneys-general-allegations).


Get Compliance Help

Some thoughts on why PCI compliance keeps going wrong in engineering organizations, what the tooling can and cannot do for you, and the one design decision that matters more than the rest of it combined.

PCI Compliance Software: Ensuring Secure Payment Transactions

pci-compliance-software-guide


Kimi.ai arrived in February 2025 with an unusual pitch: a very long context
window, real-time web search, and free unlimited access. That last part is what
got people's attention, and it's also the part most likely to be temporary.

I spent some time putting it next to ChatGPT and Claude, less to crown a winner
than to work out which of the differences are structural and which are just
where the pricing happens to sit this quarter.

## The shape of each one

|                     | Kimi.ai                               | ChatGPT                             | Claude                                           |
| ------------------- | ------------------------------------- | ----------------------------------- | ------------------------------------------------ |
| **Context window**  | ~200K characters                      | ~32K characters per prompt          | Long-context, handles detailed instructions well |
| **Modalities**      | Text, images, code                    | Primarily text, some visual support | Text and code, no image input at the time        |
| **Live web access** | Yes — searches across 1,000+ sites    | Yes, via browsing mode              | No, training data only                           |
| **Reasoning style** | Explicit chain-of-thought, RL-trained | Strong, less visibly stepwise       | Strong, especially on non-code tasks             |
| **Cost**            | Free, unlimited                       | Tiered subscription                 | Subscription                                     |

Two caveats before anyone treats this as a scorecard. These figures come from
each vendor's own materials at the time of writing, and this space moves fast
enough that context windows in particular get revised upward every few months.
Treat the table as a snapshot of February 2025, not a standing fact.

## Where the differences actually bite

**Context length is the one that changes what you can build.** Everything else
on that table is a matter of degree; context is a hard wall. If your workload is
"read this 300-page contract and answer questions about clause interactions," a
model that can hold the whole thing beats a smarter model that has to be fed
chunks. Chunking isn't just inconvenient — it silently loses cross-references,
and you often can't tell when it has.

**Live web access matters less than people expect, and more than people
expect.** Less, because for most tasks the model's training data is fine and
retrieval adds latency plus a new failure mode. More, because the tasks where it
matters — anything about current events, prices, versions, or documentation —
are exactly the ones where a confidently wrong answer is most damaging. A model
that cannot browse will not tell you it's out of date. It will just answer.

**Visible chain-of-thought is a debugging feature.** Kimi's stepwise output is
useful less because it improves accuracy and more because when the answer is
wrong you can see where the reasoning turned. If you're building on top of a
model, that observability is worth real money.

**Free is a strategy, not a property.** Unlimited free access to a frontier-ish
model is a customer acquisition cost someone is paying. Build a product
dependency on it and you're exposed to whenever that calculus changes. This is
not an argument against using it; it's an argument against assuming the terms
hold.

## Picking one for actual work

| If your workload is…                    | The deciding factor                      | Reasonable pick            |
| --------------------------------------- | ---------------------------------------- | -------------------------- |
| Long documents, whole-corpus questions  | Context window                           | Kimi.ai                    |
| Anything needing current information    | Live retrieval                           | Kimi.ai or ChatGPT         |
| Writing quality, nuance, tone           | Output quality on non-code text          | Claude                     |
| Code with sustained back-and-forth      | Instruction-following over long sessions | Claude                     |
| Broad general-purpose, widest ecosystem | Tooling and integrations                 | ChatGPT                    |
| Cost-sensitive experimentation          | Price                                    | Kimi.ai, with an exit plan |

The honest summary is that these models are closer to each other than the
marketing suggests, and the right choice usually falls out of one constraint
rather than an overall ranking. Figure out which single property your workload
actually depends on — context, freshness, writing quality, or cost — and the
decision mostly makes itself.

What Kimi's arrival really signals is that the interesting competition has moved
from raw capability to the surrounding envelope: how much you can put in, how
current it is, what it costs. That's a healthier place for the competition to
be, and it's good news for anyone building on these things rather than selling
them.

---

Evaluating models for a specific workload and want a second opinion on the
tradeoffs? [Happy to talk it through](/#contact).


Get AI Consultation

Kimi.ai showed up in February 2025 with a very long context window, live web search, and no price tag. A side-by-side look at where it lands against ChatGPT and Claude, and which differences actually change what you can build.

Notes from the build, not the brochure.

Featured

The Lottery Ticket Hypothesis: Sparse Networks That Actually Train

All posts

OpenAI Codex Goes Open Source with Ollama Integration

The Lottery Ticket Hypothesis: Sparse Networks That Actually Train

Korea's AI Blitz: Four Frontier Models, One Week

PCI Compliance Software: Ensuring Secure Payment Transactions

Kimi.ai vs. ChatGPT vs. Claude: A Comparative Analysis of Leading AI Models