AI Metrology: Ensuring the Reliability of Artificial Intelligence
Dr. Ram Sriram discussed AI metrology's role in making AI reliable, especially in healthcare. He highlighted NIST's efforts on standards, challenges like AI ethics, and the need for collaboration to ensure trustworthy, effective AI in healthcare.
- What We Have Done -
Key Takeaways
- AI Metrology Concepts: Measuring AI system's performance, trustworthiness, and reliability through standards and metrics.
- AI in Healthcare: Applications in diagnosis, treatment planning, and medical devices like continuous glucose monitoring and sepsis detection. Challenges like uncertainty quantification, adversarial attacks, and ethical concerns were addressed.
- Standards Development: NIST’s work on creating global standards for AI, emphasizing adaptability, interoperability, and reproducibility.
- Large Language Models (LLMs): Challenges in evaluating LLMs for healthcare, such as hallucination and privacy issues, and developing metrics for their performance.
- Dr. Sriram emphasized the importance of collaboration among academia, industry, and government to prioritize and develop AI standards that cater to diverse healthcare needs. The session concluded with a Q&A, addressing audience queries on AI governance, metrology, and innovation opportunities
