An OCI Architect's First Walk Through Generative AI

Every enterprise architect eventually gets that request from leadership: "Can we use AI on our data — without shipping it off to some third-party SaaS we don't control?" For me, that request landed on a Tuesday, and it sent me straight into the OCI Console, into a service I'd only skimmed the docs on: OCI Generative AI, tucked under Analytics & AI.

How Generative AI Actually Works

The intro walkthrough boiled the whole service down to three steps, and honestly, it's the cleanest mental model I've seen for explaining LLMs to a room full of database people:

1. Input — you provide text, examples, or instructions in natural language.
2. Analysis — the service processes that input and generates, summarizes, transforms, extracts, or classifies the text.
3. Output — you get a response back in the format you asked for.

Beyond the Pretrained Models: Fine-Tuning

The walkthrough didn't stop at "use what Oracle gives you." It also explained the path to a custom model: pick a pretrained base model, supply your own training file, and let OCI Generative AI fine-tune it for you — but only on dedicated AI clusters that belong exclusively to your tenancy. And fine-tuning is only half the story: once you have a custom model, you still need to create an endpoint and host that model on a dedicated cluster before anything can call it.

OCI Generative AI offers the following pretrained foundational models. Review the key features, regions, on-demand and dedicated AI cluster offerings, deprecation and retirement dates, and benchmarks for the models.

Ref: https://docs.oracle.com/en-us/iaas/Content/generative-ai/pretrained-models.htm

Two Doors: Chat and Embedding

The Playground, I learned, isn't one tool — it's two modes wearing the same UI:

• Chat lets you ask questions and get conversational responses. The chat models hold context across the conversation, so you can ask follow-ups, and you control format, length, and tone.
• Embedding converts text into vector representations — the raw material for semantic search, text classification, and clustering.

A few things I'll be repeating in the design review:

1. **Pretrained models get you started for free** — on-demand, no cluster required — but production fine-tuning and hosting custom models both require dedicated AI clusters scoped to your tenancy.

2. **Watch the lifecycle labels.** The model picker flags deprecated models inline; it's worth building a habit of checking that before locking a model into an application.

3. **Chat and Embedding are two ends of the same pipeline.** Chat is the conversational front door; Embedding is the semantic index underneath it. Most real projects need both.

4. **The console mirrors infrastructure thinking a DBA already has** — clusters, capacity, endpoints, versioning — which makes this a much shorter ramp-up for an OCI architect than the "AI" branding might suggest.