The AI Development Life Cycle: A Risk Professional's Guide

The 8 Phases of the AI-DLC

1. Defining the Problem. Every AI project starts with defining what the system is supposed to accomplish. Vague problem definitions lead to misaligned models with unpredictable behaviors. From a risk perspective: Was the use case clearly scoped? Who signed off?

2. Data Collection. AI models are only as good as the data they're trained on. Incomplete, biased, or poorly sourced training data will produce flawed outputs: often in ways that aren't obvious until production. Key risk question: Where does the data come from, and how was it validated?

3. Preparing Data. Raw data is rarely ready for training. Cleaning, labeling, and normalizing data is labor-intensive and error-prone. Poorly prepared data is a root cause of biased or inaccurate AI outputs.

4. Model Design. Engineers select the algorithm architecture best suited to the task. The choices made here: type of model, number of parameters, training approach, have long-term implications for explainability, bias risk, and computational cost.

5. Model Training. The model is repeatedly exposed to data, adjusting its internal parameters to minimize prediction errors. This phase is computationally intensive and can introduce drift or unintended behaviors if the training dataset is unrepresentative.

6. Model Evaluation. The trained model is tested on a held-out dataset it has never seen before, using metrics such as accuracy, precision, recall, and F1 score. Models that look good in testing can still fail in production.

7. Model Deployment. The model goes live. Deployment environments: cloud, on-premises, or edge : introduce new security considerations. Containerization, load balancing, and access controls all matter here.

8. Model Monitoring. This is where most organizations underinvest. Deployed models drift over time as real-world data shifts away from training data. Continuous monitoring is essential, and it's directly relevant to your TPRM continuous monitoring program.

The TPRM Connection

When a vendor uses or sells AI, they should be operating within a version of this life cycle. Your due diligence questions should probe each phase: How do they collect and validate training data? What's their model evaluation process? How do they detect and respond to model drift in production?

Coming up next: AI hallucinations: the risk nobody's favorite topic, but everyone needs to understand.