AI & Quality Readiness Scorecard
A self-assessment of up to 16 questions covering 7 key dimensions of AI readiness, informed by industry standards and frameworks: AI Adoption & Readiness, Govern & Manage AI Risk, Build AI Systems, Quality & Testing, Secure AI, Measure Delivery, and Laws & Obligations.
What this scorecard measures
The assessment covers 7 dimensions of AI and quality readiness. Below is every question and the answer options, so you can see exactly what is assessed before you start.
AI Adoption & Readiness
AI use. How widely does your team use AI tools in daily work (generative AI assistants and, if you have them, AI agents)?
- Rarely or never
- Some people, for selected tasks
- Most people, regularly, with shared good practices
AI literacy. What best describes AI literacy training for your organization's own staff?
- No training for staff
- Training mainly for technical teams
- Role-based training for staff across the organization
Govern & Manage AI Risk
AI governance. How formalized are your AI governance practices (inventory of systems, owners, and responsible use policy)? (for example, practices similar to those in ISO/IEC 42001)
- No: None
- Partial: Incomplete inventory and/or basic policy
- Complete: Full inventory (including employee AI agents/tools), assigned owners, and reviewed policy
AI risk management. Do you identify, assess, and treat the risks of your AI systems (including AI agents) using a defined process? (for example, structured approaches similar to the NIST AI RMF or ISO/IEC 23894)
- No: No systematic identification or treatment of AI risks.
- Informal: Occasional or ad-hoc risk checks, without a defined process.
- Structured: Defined process to identify, assess, and treat risks across AI systems.
Build AI Systems
AI requirements engineering. How does your team manage requirements for AI systems? (for example, using practices such as those in IREB CPRE)
- Informal: Ad-hoc or informal documentation, without a structured approach.
- Partial: Structured practices used inconsistently across projects or teams.
- Standardized: Systematic structured requirements practices applied across projects.
AI life cycle and architecture. Do you define the life cycle and document the architecture of your AI systems, including AI agents (their tools, memory, and approval points)? (for example, practices aligned with ISO/IEC 5338 or ISO/IEC/IEEE 42010)
- No: No defined life cycle or documented architecture.
- Partial: Defined for some projects or systems only.
- Complete: Defined life cycle and documented architecture views for all AI systems.
Secure development. Do you apply secure software development practices both when using AI-assisted coding and when building AI components or systems? (for example, practices aligned with the NIST SSDF)
- No: Secure development practices are not applied.
- Partial: Applied in some situations or projects, but not consistently.
- Systematic: Systematically applied (including automated checks where possible) for both AI-assisted coding and AI systems.
Quality & Testing
AI testing skills. What percentage of your team has AI testing skills (for example, skills aligned with ISTQB AI Testing or Generative AI for Testing), even without formal certification?
- Fewer than 25%: Only a small portion of the team has these skills.
- 25% to 75%: A moderate portion of the team has these skills.
- More than 75%: Most or all of the team has these skills.
Test process maturity. To what extent is your test process defined, measured, and continuously improved using a structured maturity model or framework? (for example, TMMi or similar)
- Ad-hoc / Informal: Testing is reactive or unstandardized, without a formal process model.
- Partial: Structured practices aligned with a maturity framework are applied in some projects or teams only.
- Standardized & Measured: A structured test process model is systematically embedded, measured, and improved across all projects.
AI quality and testing. Before an AI system or agent is put into use (built or bought), do you define its quality criteria and test it with a risk-based approach? (for example, practices aligned with ISO/IEC 25059 or ISO/IEC TS 42119-2)
- No: No defined quality characteristics or test approach.
- Informal: Quality and testing are handled informally or inconsistently.
- Defined: Defined quality criteria and a risk-based test approach for each AI system (built or purchased).
Secure AI
LLM and agent security. Does your team know and apply security practices for LLM applications and AI agents (for example, against prompt injection and sensitive data leaks)? (for example, practices similar to the OWASP Top 10 for LLM Applications and for Agentic Applications)
- No awareness: No awareness of specific security risks for LLMs or agents.
- Basic knowledge: Some awareness, but practices are not consistently applied.
- Applied: Security practices are applied in design, testing, and reviews.
Security management. Do you manage information security with a recognized framework or standard (for example, ISO/IEC 27001 or the NIST Cybersecurity Framework), and does it cover your AI projects, including what AI agents can access and the systems they run on?
- No: No recognized security framework or standard in use.
- Partial: Framework exists, but AI projects and agents are not covered.
- Yes: Framework is in place and explicitly covers AI projects and agents.
Measure Delivery
Delivery speed and stability. Do you track software delivery speed and stability (the DORA metrics), so you can see how AI affects them?
- None: We do not track delivery speed or stability.
- Partial: We track some metrics (for example, lead time or deployment frequency), informally or without following trends.
- Regular: We regularly track lead time, deployment frequency, change failure rate, and time to restore service, and we follow trends over time.
Impact measurement. Do you measure the effect of AI on your work and outcomes—such as time saved, quality, cost, or business results (for example, NPS, ROI, margin)—against a baseline?
- No: We do not measure the impact of AI.
- Occasionally: We measure occasionally or informally, without a consistent baseline.
- Regularly: We measure regularly against a defined baseline.
Laws & Obligations
Regulations. Do you know which AI regulations apply to your organization and use cases, including AI agents (for example, the EU AI Act if you operate in or sell to the EU), and does someone own that follow-up?
- No
- Partly, with no owner
- Yes, with a named owner and regular reviews
Data control. Do you have a strategy to keep company IP and sensitive data within controlled environments when using AI tools?
- No
- Basic policy
- Policy plus technical controls