Austrian Artificial Intelligence Podcast Manuel Pasieka

## Summary

Today on the show I am talking to Veronika Vishnevskaia. Solution Architect at ONTEC where she specialises in building RAG based Question-Answering systems.



Veronika will provide a deep dive into all relevant steps to build a Question-Answering system. Starting from data extraction and transformation, followed by text embedding, chunking and hybrid retrieval to strategies and last but not least methods to mitigate hallucinations of LLMs during the answer creation.



## AAIP Community

Join our discord server and ask guest directly or discuss related topics with the community.

https://discord.gg/5Pj446VKNU



## TOC

00:00:00 Beginning

00:03:33 Guest Introduction

00:08:51 Building Q/A Systems for businesses

00:16:27 RAG: Data extraction & pre-processing

00:26:08 RAG: Chunking & Embedding

00:36:13 RAG: Information Retrieval

00:48:59 Hallucinations

01:02:21 Future RAG systems



## Sponsors

- Quantics: Supply Chain Planning for the new normal - the never normal - https://quantics.io/

- Belichberg GmbH: Software that Saves the Planet: The Future of Energy Begins Here - https://belichberg.com/



### References

Veronika Vishnevskaia - https://www.linkedin.com/in/veronika-vishnevskaia/

Ontec - www.ontec.at

Review Hallucination Mitigation Techniques: https://arxiv.org/pdf/2401.01313.pdf

Aleph-Alpha: https://aleph-alpha.com/de/technologie/

# Summary

Today on the show I am talking to Manuel Reinsperger, Cybersecurity Expert and Penetration Tester. Manuel will provide us an introduction into the topic of Machine Learning Security with an emphasis on Chatbot and Large Language Model security.



We are going to discuss topics like AI Red Teaming that focuses on identifying and testing AI systems within an holistic approach for system security. Another major theme of the episode are different Attack Scenarios against Chatbots and Agent systems.



Manuel will explain to use, what Jailsbreak are and methods to exfiltrate information and cause harm through direct and indirect prompt injection.



Machine Learning security is a topic I am specially interested in and I hope you are going to enjoy this episode and find it useful.



## AAIP Community

Join our discord server and ask guest directly or discuss related topics with the community.

https://discord.gg/5Pj446VKNU



## TOC

00:00:00 Beginning

00:02:05 Guest Introduction

00:05:16 What is ML Security and how does it differ from Cybersecurity?

00:25:56 Attacking chatbot systems

00:41:12 Attacking RAGs with Indirect prompt injection

00:54:43 Outlook on LLM security





## Sponsors

- Quantics: Supply Chain Planning for the new normal - the never normal - https://quantics.io/

- Belichberg GmbH: Software that Saves the Planet: The Future of Energy Begins Here - https://belichberg.com/



## References

Manuel Reinsperger - https://manuel.reinsperger.org/

Test your prompt hacking skills: https://gandalf.lakera.ai/

Hacking Bing Chat: https://betterprogramming.pub/the-dark-side-of-llms-we-need-to-rethinInjectGPT: k-large-language-models-now-6212aca0581a

AI-Attack Surface: https://danielmiessler.com/blog/the-ai-attack-surface-map-v1-0/

https://blog.luitjes.it/posts/injectgpt-most-polite-exploit-ever/

https://github.com/jiep/offensive-ai-compilation

AI Security Reference List: https://github.com/DeepSpaceHarbor/Awesome-AI-Security

Prompt Injection into GPT: https://kai-greshake.de/posts/puzzle-22745/

## Summary

At the end of last year, the EU-AI Act was finalized and it spawned many discussions and a lot of doubts about the future of European AI companies.



Today on the show Peter Jeitschko, founder of JetHire an AI based recruiting platform that uses Large Language models to help recruiters find and work with candidates, talks about this perspective on the AI-Act.



We talk about the impact of the EU AI-Act on their platform, and how it falls into a high-risk use-case under the new regulation. Peter describes how the AI-Act forced them to create their company in the US and what he believes are the downsides of the EU regulation.



He describes his experience, that the EU regulations hinder innovation in Austria and Europe and how it increases legal costs and uncertainty, resulting in decision makers shying away in building and applying modern AI systems.



I think this episode is valuable for decision makers and founders of AI companies, that are affected by the upcoming AI Act and struggle to make sense of it.



## AAIP Community

Join our discord server and ask guest directly or discuss related topics with the community.

https://discord.gg/5Pj446VKNU



## TOC

00:00:00 Beginning

00:03:09 Guest Introduction

00:04:45 A founders perspective on the AI Act

00:13:45 JetHire - A recruiting platform affected the the AI Act

00:19:58 Achieving regulatory goals with good engineering

00:35:22 The mismatch between regulations and real world applications

00:48:12 European regulations vs. global AI services



## Sponsors

- Quantics: Supply Chain Planning for the new normal - the never normal - https://quantics.io/

- Belichberg GmbH: Software that Saves the Planet: The Future of Energy Begins Here - https://belichberg.com/



## References

Peter Jeitschko - https://www.linkedin.com/in/pjeitschko/

Peter Jeitschko - https://peterjeitschko.com/

JetHire - https://jethire.ai/

https://www.holisticai.com/blog/requirements-for-high-risk-ai-applications-overview-of-regulations

# Summary

For the last two years AI has been flooded with news about LLMs and their successes, but how many companies are actually making use of them in their products and services?

Today on the show I am talking to Markus Keiblinger, Managing partner of Texterous. A startup that focus on building custom LLM Solutions to help companies automate their business.

Markus will tell us about his experience when talking and working with companies building such LLM focused solutions.

Telling us about the expectations companies have on the capabilities of LLMs, as well on what companies need to have in order to be successfully implementing LLM projects.

We will discuss how Textorous has successfully focused on Retriever Augmented Generation (RAG) use cases.

RAGs is a mechanism that makes it possible to provide information to an LLM in a controlled menner, so the LLM can answer questions or follow instructions making use of that information. This enables companies to make use of their data to solve problems with LLMs, without having to train or even fine-tune models. On the show, Markus will tell us of one of these RAG projects and we will contrast building a RAG system based on Service Provider offerings like OpenAI or self hosted open source alternatives.

Last but not least, we talk about new use cases emerging with multi-modal Models, and the long term perspective that exists for custom LLM Solutions Providers like them in focusing on building integrated solutions.



## AAIP Community

Join our discord server and ask guest directly or discuss related topics with the community.

https://discord.gg/5Pj446VKNU



## TOC

00:00:00 Beginning

00:03:31 Guest Introduction

00:06:40 Challenges of applying AI in medical applications

00:17:56 Homogeneous Ensemble Methods

00:25:50 Combining base model predictions

00:40:14 Composing Ensembles

00:52:24 Explainability of Ensemble Methods



## Sponsors

- Quantics: Supply Chain Planning for the new normal - the never normal - https://quantics.io/

- Belichberg GmbH: Software that Saves the Planet: The Future of Energy Begins Here - https://belichberg.com/



### References

- Markus Keiblinger: https://www.linkedin.com/in/markus-keiblinger

- Texterous: https://texterous.com

- Book: Conversations Plato Never Captured - but an AI did: https://www.amazon.de/Conversations-Plato-Never-Captured-but/dp/B0BPVS9H9R/

## Summary

Hello and welcome back to the Austrian Artificial Intelligence Podcast in 2024.



With this episode we start into the third year of the podcast. I am very happy to see that the number of listeners has been growing steadily since the beginning and I want to thank you dear listeners for coming back to the podcast and sharing it with your friends.

Gabriel is a Bioinformatician at the Austrian Institute of Technology and is going to explain his work on ensemble methods and their application in the medical domain.

For those not familiar with the term, an Ensemble is a combination of individual base models that are combined with the goal to outperform each individual model.

So the basic idea is, that one combines multiple models that each have their strength and weaknesses into a single ensemble that in the best case has all the strengths without the weaknesses.

We have seen one type of ensemble methods in the past. These where homogeneous ensemble methods like federated learning, where one trains the same algorithm multiple times by multiple parties or different subsets of the data, for performance reasons or in order to combine model weights without sharing the training data.



Today, Gabriel will talk about heterogeneous ensembles that are a combination of different models types and their usage in medical applications. He will explain how one can use them to increase the robustness and the accuracy of predictions. We will discuss how to select and create compositions of models, as well how to combine the different predictions of the individual base models in smart ways that improve their accuracy over simply methods like averaging over majority voting.



## AAIP Community

Join our discord server and ask guest directly or discuss related topics with the community.

https://discord.gg/5Pj446VKNU



## TOC

00:00:00 Beginning

00:03:31 Guest Introduction

00:06:40 Challenges of applying AI in medical applications

00:17:56 Homogeneous Ensemble Methods

00:25:50 Combining base model predictions

00:40:14 Composing Ensembles

00:45:57 Explainability of Ensemble Methods



## Sponsors

- Quantics: Supply Chain Planning for the new normal - the never normal - https://quantics.io/

- Belichberg GmbH: Software that Saves the Planet: The Future of Energy Begins Here - https://belichberg.com/



## References

Gabriel Alexander Vignolle - https://www.linkedin.com/in/gabriel-alexander-vignolle-385b141b6/

Publications - https://publications.ait.ac.at/en/persons/gabriel.vignolle

Molecular Diagnostics - https://molecular-diagnostics.ait.ac.at/

# Summary

I am sure that most of you are familiar with the training paradigm of supervised and unsupervised learning. Where in the case of supervised learning one has a label for each training datapoint and in the unsupervised situation there are no labels.

Although there can be exceptions, everyone is well advise to perform supervised training when ever possible. But where to get those labels for your training data if traditional labeling strategies, like manual annotations are not possible?

Well often you might not have perfect labels for your data, but you have some idea what those labels might be.

And this, my dear listener is exactly the are of weak supervision.

Today on the show I am talking to Andreas Stephan who is doing is PhD in Natural Language Processing at the University of Vienna in the Digital Text Sciences group led by Professor Benjamin Roth.

Andreas will explain about his recent research in the area of weak supervision as well how Large Language Models can be used as weak supervision sources for image classification tasks.



# TOC

00:00:00 Beginning

00:01:38 Weak supervision a short introduction (by me)

00:04:17 Guest Introduction

00:08:48 What is weak supervision?

00:16:02 Paper: SepLL: Separating Latent Class Labels from Weak Supervision Noise

00:26:28 Benefits of priors to guide model training

00:29:38 Data quality & Data Quantity in training foundation models

00:36:10 Using LLM's for weak supervision

00:46:51 Future of weak supervision research



# Sponsors

- Quantics: Supply Chain Planning for the new normal - the never normal - https://quantics.io/

- Belichberg GmbH: We do digital transformations as your innovation partner - https://belichberg.com/



# References

- Andreas Stephan - https://andst.github.io/

- Stephan et al. "SepLL: Separating Latent Class Labels from Weak Supervision Noise" (2022) - https://arxiv.org/pdf/2210.13898.pdf

- Gunasekar et al. "Textbooks are all you need" (2023) - https://arxiv.org/abs/2306.11644

- Introduction into weak supervision: https://dawn.cs.stanford.edu/2017/07/16/weak-supervision/