OpenAI API: Models, Responses, and Tooling

Imagine you are building a note-taking app.

You want one button that turns a messy meeting note into a clean summary.

You have two options.

Option A: train your own large language model. Costs millions of dollars. Needs thousands of GPUs and a team of researchers. Years of work.

Option B: rent one. Somebody else already trained excellent models. They run on servers the provider owns. You send text over the internet, the service sends an answer back, and you pay only for what you use.

The OpenAI API is the most famous version of option B — so famous that its request shape ("send a list of messages, get a completion back") became the industry standard that nearly every other provider copies on purpose. That makes it the surface every AI engineer should learn first.

The whole contract in one line:

A token is the billing unit of text — roughly a word fragment. A handy rule: 100 tokens is about 75 words. Everything in this API — cost, speed, limits, caching — is measured in tokens.

So the questions this topic answers, one by one:

• Which model do I order, and what is on the menu? (the lineup)
• Do I re-send the whole conversation every turn, or can the server remember for me? (Chat Completions vs the Responses API)
• How do I make the AI do things, not just talk? (function calling)
• How do I make its answer machine-readable every single time? (structured outputs)
• How do I stop paying full price for prompt parts that never change? (prompt caching)
• How do I run 200k jobs overnight at half price? (Batch API)
• What does operating this at real scale feel like? (limits, tiers, keys)