(Read time: 5 minutes)

We talk a lot about what AI can do. Write, predict, generate.

But how does it actually learn?

This lesson walks you through the step-by-step process of training an AI model—from collecting data to testing predictions—so you can understand how systems like GPT are built from the ground up.

At its core, an AI model is a series of numbers called “parameters” (or “weights”). When billions of these parameters are organized just right, they create a model.

We’ll walk through the process from start to finish:

1. Data Collection and Preparation

2. Pre-Training

3. Fine-Tuning

4. Testing

🧪 Step 1: Data Collection and Preparation

Imagine teaching a child to recognize animals. You’d show them tons of pictures and say, “This is a cat,” “This is a dog.”

AI works the same way.

• Data Collection: Gather massive amounts of text, images, or audio. This is the model’s “study material.”

• Data Preparation: Clean it up. Remove duplicates, errors, and noise so the model isn’t learning from bad data.

💡 Crypto Tie-In: High-quality data is AI’s most valuable resource—and in decentralized systems, contributors can earn tokens for supplying domain-specific datasets (think: medical images, legal docs, trading histories, etc.).

⚙️ Step 2: Pre-Training

Now comes the heavy lifting.

In pre-training, the AI model uses neural networks to identify patterns in vast datasets, building a foundational understanding of language, structure, and logic.

This process is extremely resource-intensive.

For large models, it can cost hundreds of millions of dollars in compute, powered by fleets of GPUs or TPUs (Tensor Processing Units) running 24/7 in cloud data centers like AWS, Google Cloud, or Azure.

Why so expensive? Because the model must analyze enormous volumes of data and adjust billions of parameters—each one a tiny dial that tunes the model’s intelligence.

Depending on the model’s size and complexity, pre-training can take days, weeks, or even months to complete.

🧠 Step 3: Fine-Tuning

Pre-training gives the model a broad education

At this stage, the model has learned general patterns, common structures, and relationships across language and data.

It’s like someone who’s read a ton of books—they understand a lot, but they’re not yet ready to apply that knowledge to specific tasks.

That’s where fine-tuning comes in.

Fine-tuning takes a pre-trained model (like Llama 3) and further trains it on a smaller, domain-specific dataset.

This sharpens the model’s ability to assist us humans: whether it’s diagnosing medical conditions, analyzing legal documents, or simply serving as a more capable everyday assistant.

💡 Crypto Tie-In: A startup could take an open-source LLM and fine-tune it using a curated medical dataset. Every time that model is used, contributors and stakers could get paid. The core idea behind Payable AI models like those built on OpenLedger.

Step 4: Testing

Now it’s time to check how well the model performs.

We feed it new data it hasn’t seen before and measure how accurate its responses are. This helps catch bugs, fix errors, and validate its capabilities.

🧠 Example: Training a Digit Recognition AI

Let’s say we want to train an AI to recognize handwritten numbers (0–9).

This tweet is a fascinating graphical representation of how the model works:

Step 1: Data Collection and Preparation

We collect thousands of images of handwritten numbers, each labelled with the correct digit. This serves as the model’s “study material.”

Step 2: Pre-Training

The model begins by analyzing these images to recognize basic shapes, like a “0” as a closed loop or a “1” as a straight line. It’s learning general patterns in the data.

Step 3: Fine-Tuning

We fine-tune it with messier, more stylized handwriting to make it more accurate. This extra step helps it adapt to real-world variations, like someone writing a “5” with a bit of a flourish.

Step 4: Testing

Finally, we test the model with new handwritten numbers. If it correctly identifies 98 out of 100 digits, we know it’s performing well!

And voilà! We have a digit-recognition model. Large language models like GPT are trained in a similar way, just with a massive amount of data and much more computing power.

What Are Parameters, Really?

You’ve heard the buzzwords: “GPT-4 has over a trillion parameters.”

But what does that mean?

Imagine a massive control panel with billions of tiny dials. Each dial represents a parameter—and adjusting those dials is how the AI “learns.”

The model uses these dials to determine, for instance, that in the sentence:

“The apple is red.”

“apple” is a noun, and “red” is describing it. More parameters = better understanding of nuance, context, and relationships.

To put it in perspective:

⚠️ Tradeoff: More parameters = more intelligence, but also more compute costs.

What’s Next?

Next, we’ll explore the core architecture that powers ChatGPT: transformer models.

If today was about how AI learns, tomorrow is about why transformers changed the game.

Cheers,

Teng Yan