Mitigating Hallucinations and Enhancing Confidence Estimation in LLMs for reliable Information Extraction Intern

Key Skills

Aartificial intelligence, ChatGPT, Large Language Models - LLMs

Job Description

10–12-week development experience running through the summer months each year

Internships are meant to be a catalyst in helping students gain hands on learning & exposure to the career they’re pursuing.

Internships benefit students by supporting them to:

• Build their professional network.
• Gain relevant experience for post-graduation opportunities.
• Expose them to AstraZeneca as a possible future employer.

Intern Eligibility Requirements:

• Students must be 18 years or older at time of hire; and not require sponsorship to work in the UK
• Students pursuing undergraduate, masters or doctoral degree.

A salary will be paid.

Large Language Models (LLMs) such as GPT have gained significant attention for their remarkable ability to generate human-like test making them indispensable tools for various natural language applications. Multiple teams within AZ are already involved in utilizing these powerful LLMs for various applications including content/report generation, summarization, chat assistant, multi-modal biomarkers, information extraction from unstructured documents etc.

However, the presence of hallucinations in LLM generated content can pose significant risk to medical decision making, patient safety and data integrity. Hence it is critical to investigate and develop methods to improve the reliability of these powerful LLMs across AZ.

This work delves into the critical importance of mitigating hallucinations in LLMs and thereby improving the interpretability of LLM responses by estimating well-calibrated confidence score for LLM predictions. The key objectives of this project are,

• Develop a method to utilize combination of expert models to provide confidence bounds at an output-level and mitigate hallucination in LLMs primarily for information extraction from unstructured and semi-structured data sources.
• Develop confidence estimation techniques that provide a clear understanding of the reliability of AI-generated responses across AZ applications.
• Conduct extensive experiments, collect, and analyse data (internal dataset available), and publish the research findings to further increase the impact.

What are we looking for?

At AstraZeneca, we put patients first and strive to meet their unmet needs worldwide. We’re deeply committed to our strategy of being a great place to work. For us, that includes fostering an environment where everyone can be among inclusive, supportive individuals with curious minds.

A company that genuinely follows the science and values individuals at all levels, you’ll be backed and encouraged to speak up, ask questions, and share ideas to push the boundaries of science and continuously learn and explore.

Our purpose is bold and so is our approach. Becoming a more agile and innovative company means building a dynamic, inspiring culture where we celebrate entrepreneurial thinking and act with a sense of urgency. We are courageous, taking risks and learning from both success and failure. We are curious, creative, and open to new ideas and ways of working. Above all, we are passionate about science and driven to always put patients first. When you join us, you will be part of a great place to work; in an environment that energizes and empowers each of us to achieve our goal to develop and deliver medicines.