Generative AI (GenAI) models start their learning from hundreds of terabytes of data from the internet. For example, ChatGPT 3.5 was trained on 175 billion parameters. This initial step is called "pretraining".

Companies like OpenAI spent months and millions of dollars on this stage. For instance, theGenAI model behind ChatGPT cost OpenAI over $100 million just in this phase!

During pretraining, models learn from websites such as Wikipedia, books, and online articles.

As a result, they can communicate in various languages, analyze data, and even write computer code.

Predict the future by learning it

You didn’t come this far to stop

Get in Touch

We'd love to hear from you about AI education!