MLOps is the discipline of treating machine learning systems the way seasoned DBAs treat database pipelines: with version control, monitoring, rollback capability, and zero tolerance for silent failures. This post walks through a production-grade MLOps architecture on Oracle Cloud Infrastructure (OCI), complete with working scripts, pipeline definitions, and the hard-won lessons that don't show up in vendor documentation.
The architecture covers three major phases:
- Data Preparation — exploration, cleansing, feature engineering in OCI Data Science
- Model Training — automated CI/CD pipeline via OCI DevOps, containerized jobs, MLFlow tracking
- Model Deployment — serving via OKE (Oracle Kubernetes Engine) with a live inference endpoint
Let's build it layer by layer.
Architecture Overview
GitHub ──(code push trigger)──► OCI DevOps Build & Deploy Pipeline
│
┌────────────────────────┼────────────────────────┐
▼ ▼ ▼
OCI DataScience OCI Container Registry OCI ObjectStorage
┌──────────────────┐ ┌──────────────────────┐ ┌──────────────────┐
│ Data Access & │ │ Training Images │ │ Datasets │
│ Exploration │ │ Inference Images │ │ Model Artifacts │
│ Model Training │ └──────────────────────┘ │ Model Backups │
│ Model Dev │ └──────────────────┘
└──────────────────┘
│
└──────────────────────────────────────────────────►
OKE / MLFlow
┌─────────────────────────┐
│ Model Evaluation │
│ Model Monitoring │
│ Model Registry │
│ Model Inference Endpoint│
└─────────────────────────┘
The bottom layer — OCI IAM, Logging, Monitoring, Secrets, VCN, Notifications — is your operational safety net that runs silently under everything else.
Phase 1 — Data Preparation in OCI Data Science
1.1 Environment Setup
Every project starts with a reproducible conda environment. Don't skip this — environment drift is the #1 cause of "but it worked on my machine" incidents.
# environment.yml
name: mlops-oci
channels:
- conda-forge
- defaults
dependencies:
- python=3.10
- pip
- pip:
- oracle-ads==2.9.1
- scikit-learn==1.3.0
- pandas==2.0.3
- mlflow==2.8.1
- oci==2.112.0
- boto3
- pyarrow
- great-expectations==0.18.0
conda env create -f environment.yml
conda activate mlops-oci
1.2 Data Access & Exploration (OCI ObjectStorage → Notebook)
# data_exploration.py
import ads
import pandas as pd
from ads.dataset.factory import DatasetFactory
from great_expectations.core import ExpectationSuite
# Authenticate using resource principal (preferred in OCI notebooks)
ads.set_auth(auth="resource_principal")
BUCKET = "mlops-datasets"
NAMESPACE = "your-tenancy-namespace"
OBJECT_NAME = "churn_raw.parquet"
# Pull dataset from Object Storage
ds = DatasetFactory.open(
f"oci://{BUCKET}@{NAMESPACE}/{OBJECT_NAME}",
format="parquet",
target="churn"
)
df = ds.to_pandas()
print(f"Shape: {df.shape}")
print(df.dtypes)
print(df.describe())
1.3 Data Quality Gate with Great Expectations
This is the DBA mindset applied to ML: validate your data before you trust it.
# data_quality.py
import great_expectations as gx
context = gx.get_context()
# Define expectations
suite = context.add_expectation_suite("churn_suite")
validator = context.get_validator(
batch_request=...,
expectation_suite_name="churn_suite"
)
# Column presence
validator.expect_table_columns_to_match_ordered_list(
column_list=["customer_id","tenure","monthly_charges","total_charges","churn"]
)
# Nullability
validator.expect_column_values_to_not_be_null("customer_id")
validator.expect_column_values_to_not_be_null("churn")
# Value ranges
validator.expect_column_values_to_be_between("tenure", min_value=0, max_value=120)
validator.expect_column_values_to_be_between("monthly_charges", min_value=0, max_value=500)
# Categorical validity
validator.expect_column_values_to_be_in_set("churn", value_set=["Yes", "No"])
results = validator.validate()
if not results["success"]:
raise ValueError(f"Data quality check FAILED:\n{results}")
print("✅ Data quality checks passed.")
1.4 Feature Engineering & Save to Object Storage
# feature_engineering.py
import pandas as pd
import ads
from ads.dataset.factory import DatasetFactory
ads.set_auth(auth="resource_principal")
def engineer_features(df: pd.DataFrame) -> pd.DataFrame:
df = df.copy()
# Encode target
df["churn_flag"] = (df["churn"] == "Yes").astype(int)
# Derived features
df["avg_monthly_spend"] = df["total_charges"] / (df["tenure"] + 1)
df["is_long_term"] = (df["tenure"] > 24).astype(int)
df["high_spender"] = (df["monthly_charges"] > 70).astype(int)
# One-hot encode
cat_cols = ["contract", "payment_method", "internet_service"]
df = pd.get_dummies(df, columns=cat_cols, drop_first=True)
# Drop originals
df.drop(columns=["churn", "customer_id"], inplace=True)
return df
df_raw = pd.read_parquet("churn_raw.parquet")
df_features = engineer_features(df_raw)
# Save engineered features back to Object Storage
output_path = "oci://mlops-datasets@your-tenancy-namespace/churn_features.parquet"
df_features.to_parquet(output_path)
print(f"✅ Feature set saved: {df_features.shape}")
Phase 2 — Model Training with OCI DevOps CI/CD
2.1 Project Structure
mlops-churn/
├── .github/ # GitHub metadata (PR templates, etc.)
├── build_spec.yaml # OCI DevOps build spec
├── deploy_spec.yaml # OCI DevOps deploy spec
├── Dockerfile.train # Training container
├── Dockerfile.inference # Inference container
├── src/
│ ├── train.py # Training entrypoint
│ ├── evaluate.py # Evaluation logic
│ ├── serve.py # FastAPI inference server
│ └── utils/
│ ├── data_quality.py
│ └── feature_engineering.py
├── k8s/
│ ├── deployment.yaml # OKE deployment manifest
│ └── service.yaml # OKE service manifest
├── environment.yml
└── requirements.txt
2.2 OCI DevOps Build Spec
# build_spec.yaml
version: 0.1
component: build
timeoutInSeconds: 1800
shell: bash
steps:
- type: Command
name: "Install dependencies"
command: |
pip install -r requirements.txt
- type: Command
name: "Run data quality checks"
command: |
python src/utils/data_quality.py
- type: Command
name: "Build Training Docker Image"
command: |
docker build \
-t ${OCI_RESOURCE_PRINCIPAL_REGION}.ocir.io/${TENANCY_NAMESPACE}/mlops-train:${OCI_BUILD_RUN_ID} \
-f Dockerfile.train .
- type: Command
name: "Push Training Image to OCIR"
command: |
docker push \
${OCI_RESOURCE_PRINCIPAL_REGION}.ocir.io/${TENANCY_NAMESPACE}/mlops-train:${OCI_BUILD_RUN_ID}
- type: Command
name: "Build Inference Docker Image"
command: |
docker build \
-t ${OCI_RESOURCE_PRINCIPAL_REGION}.ocir.io/${TENANCY_NAMESPACE}/mlops-infer:${OCI_BUILD_RUN_ID} \
-f Dockerfile.inference .
- type: Command
name: "Push Inference Image to OCIR"
command: |
docker push \
${OCI_RESOURCE_PRINCIPAL_REGION}.ocir.io/${TENANCY_NAMESPACE}/mlops-infer:${OCI_BUILD_RUN_ID}
outputArtifacts:
- name: train_image_tag
type: BINARY
location: ${OCI_BUILD_RUN_ID}
2.3 Training Script with MLFlow Tracking
# src/train.py
import os
import mlflow
import mlflow.sklearn
import pandas as pd
import numpy as np
from sklearn.ensemble import GradientBoostingClassifier
from sklearn.model_selection import train_test_split, cross_val_score
from sklearn.metrics import (
classification_report, roc_auc_score,
precision_score, recall_score, f1_score
)
import ads
import joblib
# ── Config ────────────────────────────────────────────────────────────────────
ads.set_auth(auth="resource_principal")
MLFLOW_TRACKING_URI = os.environ["MLFLOW_TRACKING_URI"]
EXPERIMENT_NAME = "churn-prediction-v2"
BUCKET = "mlops-datasets"
NAMESPACE = os.environ["OCI_TENANCY_NAMESPACE"]
MODEL_BUCKET = "mlops-artifacts"
mlflow.set_tracking_uri(MLFLOW_TRACKING_URI)
mlflow.set_experiment(EXPERIMENT_NAME)
# ── Load Features ─────────────────────────────────────────────────────────────
df = pd.read_parquet(
f"oci://{BUCKET}@{NAMESPACE}/churn_features.parquet"
)
X = df.drop(columns=["churn_flag"])
y = df["churn_flag"]
X_train, X_test, y_train, y_test = train_test_split(
X, y, test_size=0.2, random_state=42, stratify=y
)
# ── Hyperparameters ───────────────────────────────────────────────────────────
params = {
"n_estimators": int(os.environ.get("N_ESTIMATORS", 200)),
"max_depth": int(os.environ.get("MAX_DEPTH", 5)),
"learning_rate": float(os.environ.get("LEARNING_RATE", 0.05)),
"subsample": float(os.environ.get("SUBSAMPLE", 0.8)),
"random_state": 42,
}
# ── Train & Track ─────────────────────────────────────────────────────────────
with mlflow.start_run() as run:
mlflow.log_params(params)
mlflow.set_tag("model_type", "GradientBoosting")
mlflow.set_tag("dataset_version", "churn_features_v2")
model = GradientBoostingClassifier(**params)
model.fit(X_train, y_train)
# Metrics
y_pred = model.predict(X_test)
y_proba = model.predict_proba(X_test)[:, 1]
metrics = {
"roc_auc": roc_auc_score(y_test, y_proba),
"precision": precision_score(y_test, y_pred),
"recall": recall_score(y_test, y_pred),
"f1_score": f1_score(y_test, y_pred),
}
# Cross-validation stability check
cv_scores = cross_val_score(model, X_train, y_train, cv=5, scoring="roc_auc")
metrics["cv_roc_auc_mean"] = cv_scores.mean()
metrics["cv_roc_auc_std"] = cv_scores.std()
mlflow.log_metrics(metrics)
print(classification_report(y_test, y_pred))
print(f"\nš Metrics: {metrics}")
# ── Promotion Gate ─────────────────────────────────────────────────────────
MIN_AUC = float(os.environ.get("MIN_AUC_THRESHOLD", 0.80))
if metrics["roc_auc"] < MIN_AUC:
raise ValueError(
f"❌ Model ROC-AUC {metrics['roc_auc']:.4f} < threshold {MIN_AUC}. "
"Halting promotion."
)
# ── Log Model & Register ───────────────────────────────────────────────────
mlflow.sklearn.log_model(
model,
artifact_path="model",
registered_model_name="churn-gbt",
input_example=X_test.head(5),
signature=mlflow.models.infer_signature(X_test, y_pred),
)
# Save artifact to Object Storage as backup
local_path = "/tmp/model.joblib"
joblib.dump(model, local_path)
import oci
object_storage = oci.object_storage.ObjectStorageClient(
config=oci.config.from_file()
)
run_id = run.info.run_id
with open(local_path, "rb") as f:
object_storage.put_object(
namespace_name=NAMESPACE,
bucket_name=MODEL_BUCKET,
object_name=f"churn-gbt/{run_id}/model.joblib",
put_object_body=f,
)
print(f"\n✅ Run ID: {run_id}")
print(f"✅ Model registered in MLFlow: churn-gbt")
print(f"✅ Artifact backed up to OCI Object Storage")
2.4 Dockerfiles
# Dockerfile.train
FROM python:3.10-slim
WORKDIR /app
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt
COPY src/ ./src/
COPY environment.yml .
ENV PYTHONUNBUFFERED=1
ENTRYPOINT ["python", "src/train.py"]
# Dockerfile.inference
FROM python:3.10-slim
WORKDIR /app
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt fastapi uvicorn
COPY src/serve.py .
EXPOSE 8080
ENTRYPOINT ["uvicorn", "serve:app", "--host", "0.0.0.0", "--port", "8080"]
Phase 3 — Model Serving on OKE with FastAPI
3.1 Inference Server
# src/serve.py
import os
import mlflow
import mlflow.sklearn
import pandas as pd
import numpy as np
from fastapi import FastAPI, HTTPException
from pydantic import BaseModel, Field
from typing import List, Optional
import logging
logging.basicConfig(level=logging.INFO)
logger = logging.getLogger(__name__)
app = FastAPI(
title="Churn Prediction API",
description="MLOps inference endpoint — OCI OKE",
version="2.0.0"
)
# ── Model Loading ──────────────────────────────────────────────────────────────
MLFLOW_TRACKING_URI = os.environ["MLFLOW_TRACKING_URI"]
MODEL_NAME = os.environ.get("MODEL_NAME", "churn-gbt")
MODEL_STAGE = os.environ.get("MODEL_STAGE", "Production")
mlflow.set_tracking_uri(MLFLOW_TRACKING_URI)
logger.info(f"Loading model: {MODEL_NAME} @ stage={MODEL_STAGE}")
model = mlflow.sklearn.load_model(f"models:/{MODEL_NAME}/{MODEL_STAGE}")
logger.info("✅ Model loaded successfully")
# ── Schemas ────────────────────────────────────────────────────────────────────
class CustomerFeatures(BaseModel):
tenure: float = Field(..., ge=0, le=120)
monthly_charges: float = Field(..., ge=0, le=500)
avg_monthly_spend: float
is_long_term: int = Field(..., ge=0, le=1)
high_spender: int = Field(..., ge=0, le=1)
contract_one_year: int = Field(0, ge=0, le=1)
contract_two_year: int = Field(0, ge=0, le=1)
class PredictionRequest(BaseModel):
customers: List[CustomerFeatures]
threshold: Optional[float] = 0.5
class PredictionResult(BaseModel):
customer_index: int
churn_probability: float
churn_prediction: bool
risk_tier: str
# ── Endpoints ──────────────────────────────────────────────────────────────────
@app.get("/health")
def health():
return {"status": "ok", "model": MODEL_NAME, "stage": MODEL_STAGE}
@app.post("/predict", response_model=List[PredictionResult])
def predict(request: PredictionRequest):
try:
df = pd.DataFrame([c.dict() for c in request.customers])
probas = model.predict_proba(df)[:, 1]
results = []
for i, prob in enumerate(probas):
tier = (
"HIGH" if prob >= 0.70 else
"MEDIUM" if prob >= 0.40 else
"LOW"
)
results.append(PredictionResult(
customer_index = i,
churn_probability = round(float(prob), 4),
churn_prediction = bool(prob >= request.threshold),
risk_tier = tier,
))
return results
except Exception as e:
logger.error(f"Prediction error: {e}")
raise HTTPException(status_code=500, detail=str(e))
@app.get("/model/info")
def model_info():
client = mlflow.tracking.MlflowClient()
mv = client.get_latest_versions(MODEL_NAME, stages=[MODEL_STAGE])[0]
return {
"name": mv.name,
"version": mv.version,
"stage": mv.current_stage,
"run_id": mv.run_id,
"description": mv.description,
}
3.2 Kubernetes Manifests
# k8s/deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: churn-inference
namespace: mlops
labels:
app: churn-inference
version: v2
spec:
replicas: 2
selector:
matchLabels:
app: churn-inference
template:
metadata:
labels:
app: churn-inference
spec:
containers:
- name: inference
image: <region>.ocir.io/<namespace>/mlops-infer:<BUILD_TAG>
ports:
- containerPort: 8080
env:
- name: MLFLOW_TRACKING_URI
valueFrom:
secretKeyRef:
name: mlops-secrets
key: mlflow-uri
- name: MODEL_NAME
value: "churn-gbt"
- name: MODEL_STAGE
value: "Production"
resources:
requests:
memory: "512Mi"
cpu: "250m"
limits:
memory: "1Gi"
cpu: "1000m"
readinessProbe:
httpGet:
path: /health
port: 8080
initialDelaySeconds: 15
periodSeconds: 10
livenessProbe:
httpGet:
path: /health
port: 8080
initialDelaySeconds: 30
periodSeconds: 20
---
# k8s/service.yaml
apiVersion: v1
kind: Service
metadata:
name: churn-inference-svc
namespace: mlops
spec:
selector:
app: churn-inference
ports:
- protocol: TCP
port: 80
targetPort: 8080
type: LoadBalancer
3.3 OKE Deploy Script
#!/bin/bash
# deploy_oke.sh
set -euo pipefail
BUILD_TAG="${1:?Usage: ./deploy_oke.sh <build_tag>}"
NAMESPACE="mlops"
REGION="us-ashburn-1"
OCIR_NAMESPACE="your-tenancy-namespace"
echo "š Deploying churn-inference:${BUILD_TAG} to OKE..."
# Login to OCIR
docker login \
${REGION}.ocir.io \
-u "${OCIR_NAMESPACE}/oracleidentitycloudservice/${OCI_USER_EMAIL}" \
-p "${OCI_AUTH_TOKEN}"
# Substitute build tag in manifests
sed "s/<BUILD_TAG>/${BUILD_TAG}/g" k8s/deployment.yaml | \
kubectl apply -n ${NAMESPACE} -f -
kubectl apply -n ${NAMESPACE} -f k8s/service.yaml
# Wait for rollout
kubectl rollout status deployment/churn-inference -n ${NAMESPACE} --timeout=5m
echo "✅ Deployment complete."
kubectl get pods -n ${NAMESPACE} -l app=churn-inference
Phase 4 — Monitoring & Governance
4.1 Model Monitoring — Drift Detection
# monitoring/drift_detector.py
"""
Scheduled job: compares production inference distributions
against training baseline. Fires OCI Notification on drift.
"""
import os
import pandas as pd
import numpy as np
from scipy.stats import ks_2samp
import oci
import json
DRIFT_THRESHOLD = 0.05 # KS statistic p-value threshold
BUCKET = "mlops-artifacts"
NAMESPACE = os.environ["OCI_TENANCY_NAMESPACE"]
def load_baseline() -> pd.DataFrame:
return pd.read_parquet(
f"oci://{BUCKET}@{NAMESPACE}/baselines/churn_train_dist.parquet"
)
def load_production_window(days: int = 7) -> pd.DataFrame:
"""Load last N days of inference logs from Object Storage."""
return pd.read_parquet(
f"oci://{BUCKET}@{NAMESPACE}/inference_logs/recent_{days}d.parquet"
)
def check_drift(baseline: pd.DataFrame, production: pd.DataFrame) -> dict:
results = {}
numeric_cols = baseline.select_dtypes(include=[np.number]).columns
for col in numeric_cols:
stat, p_value = ks_2samp(
baseline[col].dropna(),
production[col].dropna()
)
results[col] = {
"ks_statistic": round(stat, 4),
"p_value": round(p_value, 4),
"drifted": p_value < DRIFT_THRESHOLD,
}
return results
def send_alert(drifted_features: list):
config = oci.config.from_file()
ons = oci.ons.NotificationDataPlaneClient(config)
topic_id = os.environ["OCI_NOTIFICATION_TOPIC_ID"]
message = json.dumps({
"alert": "DATA DRIFT DETECTED",
"features": drifted_features,
"action": "Review and retrain churn-gbt model",
}, indent=2)
ons.publish_message(
topic_id=topic_id,
message_details=oci.ons.models.MessageDetails(
title="MLOps Alert: Model Drift",
body=message,
)
)
print(f"šØ Alert sent for features: {drifted_features}")
if __name__ == "__main__":
baseline = load_baseline()
production = load_production_window(days=7)
drift_report = check_drift(baseline, production)
drifted = [f for f, v in drift_report.items() if v["drifted"]]
if drifted:
print(f"⚠️ Drift detected in: {drifted}")
send_alert(drifted)
else:
print("✅ No significant drift detected.")
4.2 Automated Retraining Trigger
# monitoring/retrain_trigger.py
"""
If drift is detected or performance degrades below threshold,
automatically kick off a new OCI DataScience Job (retraining run).
"""
import os
import oci
def trigger_retraining_job(reason: str):
config = oci.config.from_file()
ds = oci.data_science.DataScienceClient(config)
job_id = os.environ["OCI_DATASCIENCE_JOB_ID"]
run = ds.create_job_run(
create_job_run_details=oci.data_science.models.CreateJobRunDetails(
project_id=os.environ["OCI_PROJECT_ID"],
compartment_id=os.environ["OCI_COMPARTMENT_ID"],
job_id=job_id,
display_name=f"auto-retrain-trigger",
job_environment_configuration_override=None,
job_configuration_override_details=oci.data_science.models.DefaultJobConfigurationDetails(
environment_variables={
"RETRAIN_REASON": reason,
"N_ESTIMATORS": "300",
"MIN_AUC_THRESHOLD": "0.82",
}
),
)
)
print(f"✅ Retraining job launched: {run.data.id}")
return run.data.id
4.3 OCI IAM Policy — Least Privilege for MLOps
# iam_policies.tf (Terraform)
resource "oci_identity_policy" "mlops_datascience" {
name = "mlops-datascience-policy"
description = "Allows DataScience resources to read/write Object Storage and publish notifications"
compartment_id = var.compartment_id
statements = [
"Allow dynamic-group mlops-dg to read object-family in compartment mlops-compartment",
"Allow dynamic-group mlops-dg to manage objects in bucket 'mlops-datasets'",
"Allow dynamic-group mlops-dg to manage objects in bucket 'mlops-artifacts'",
"Allow dynamic-group mlops-dg to use ons-topics in compartment mlops-compartment",
"Allow dynamic-group mlops-dg to manage data-science-jobs in compartment mlops-compartment",
"Allow dynamic-group mlops-dg to manage data-science-job-runs in compartment mlops-compartment",
"Allow dynamic-group mlops-dg to read secret-bundles in vault id ${var.vault_id}",
]
}
Phase 5 — End-to-End Integration Test
# tests/e2e_test.py
import requests
import json
ENDPOINT = "http://<OKE_LOAD_BALANCER_IP>/predict"
payload = {
"threshold": 0.5,
"customers": [
{
"tenure": 3,
"monthly_charges": 85.0,
"avg_monthly_spend": 21.25,
"is_long_term": 0,
"high_spender": 1,
"contract_one_year": 0,
"contract_two_year": 0,
},
{
"tenure": 48,
"monthly_charges": 45.0,
"avg_monthly_spend": 44.1,
"is_long_term": 1,
"high_spender": 0,
"contract_one_year": 0,
"contract_two_year": 1,
},
]
}
response = requests.post(ENDPOINT, json=payload)
response.raise_for_status()
results = response.json()
for r in results:
print(
f"Customer {r['customer_index']} | "
f"P(churn)={r['churn_probability']:.2%} | "
f"Tier={r['risk_tier']} | "
f"Prediction={'CHURN' if r['churn_prediction'] else 'RETAIN'}"
)
Expected output:
Customer 0 | P(churn)=78.32% | Tier=HIGH | Prediction=CHURN
Customer 1 | P(churn)=12.14% | Tier=LOW | Prediction=RETAIN
Key Lessons — The DBA Perspective
| DBA Principle | MLOps Translation |
|---|---|
| Always version your schema | Always version your features and training data |
| Never trust incoming data | Run Great Expectations before every training run |
| Keep backups | Store every model artifact to Object Storage |
| Monitor query performance | Monitor model drift and prediction latency |
| Use least-privilege IAM | Scope resource principal policies tightly |
| Have a rollback plan | Keep previous MLFlow model versions in Staging |
| Automate your maintenance jobs | Automate retraining triggers on drift events |
Deployment Checklist
- [ ] OCI Vault secrets configured (MLFlow URI, OCIR auth token)
- [ ] Dynamic group and IAM policies applied
- [ ] GitHub repo connected to OCI DevOps project
- [ ] Build pipeline validated with
build_spec.yaml - [ ] Training job baseline run completed and logged to MLFlow
- [ ] Model promoted to
Productionstage in MLFlow Registry - [ ] OKE namespace and secrets created (
kubectl create secret) - [ ] Kubernetes manifests deployed and readiness probes passing
- [ ] Drift monitoring job scheduled (OCI Scheduler or cron)
- [ ] OCI Notification topic wired to ops team email/PagerDuty
- [ ] End-to-end test passing against live inference endpoint
References
- OCI Data Science Documentation
- OCI DevOps Build Pipelines
- MLFlow Model Registry
- Oracle Accelerated Data Science (ADS) SDK
- OKE Getting Started
- Great Expectations
| DBA Principle | MLOps Translation |
| Schema Integrity | Data Quality Gates (Great Expectations) |
| ACID Compliance | Versioned Models & Data (MLFlow/Object Storage) |
| Performance Tuning | Hyperparameter Optimization |
| Point-in-Time Recovery | Model Rollback via MLFlow Stages |
| Audit Logs | OCI Logging & MLFlow Run Tracking |
