The AI team take a deep dive into Tom Gilb’s presentation to the BCS in July 2024 on AI quality which advocates for a multidimensional approach to evaluating AI systems, moving beyond vague descriptions. He argues that current AI models are defective due to inherent instability and a lack of transparency. Gilb proposes using quantifiable metrics for qualities like transparency and security, incorporating a stakeholder model for a more comprehensive understanding. The presentation emphasizes the need for measurable standards to assess AI systems effectively, contrasting Large Language Models (LLMs) with the potential of Artificial General Intelligence (AGI).
The presentation slides in pdf format may be downloaded from https://www.dropbox.com/scl/fo/u1fisxhd5cyrp7iij2i98/AAYYSATm4Yt40X0t8lEJdpQ?rlkey=3n4nn6077nig6gvo5qiuu0em8&dl=0