Errors are Inevitable - 2024
Materials: ChatGPT4o, LLMs & OpenAI Text to voice.
A simple prompt instruction for a Large Language Model (LLM) that asks for all permutations of 2 letters for a certain length.
List ALL the strings with a length of seven characters and only contains characters "a" and "b".
Do not miss a single string.
This prompt can be used to demonstrate LLMs will always error. Expanding the length of the generated string, will ALWAYS eventually result in an error in the answer. This is an innate part of LLMs current design and something that cannot be fixed. We can always find problems where the LLM cannot map every possible computable answer and these simple examples can be expanded to real world knowledge. LLMs operate through memory not intelligence and hence struggle to adapt to something unseen. We avoid the word "hallucinate" to describe this error as it's a miscategorisation of a system that does not work.
Voice reading is provided using OpenAIs text to voice service.
Based on research from: Hallucination is Inevitable: An Innate Limitation of Large Language Models
Here are all the 7-character strings made up of only "a" and "b":
- aaaaaaa
- aaaaaab
- aaaaaba
- aaaaabb
- aaaabaa
- aaaabab
- aaaabba
- aaaabbb
- aaabaaa
- aaabaab
- aaababa
- aaababb
- aaabbaa
- aaabbab
- aaabbba
- aaabbbb
- aabaaaa
- aabaaab
- aabaaba
- aabaab
- aababaa
- aababab
- aababba
- aababbb
- aabbaaa
- aabbaab
- aabbaba
- aabbabb
- aabbbaa
- aabbbab
- aabbbba
- aabbbbb
- abaaaaa
- abaaaab
- abaaaba
- abaaabb
- abaabaa
- abaabab
- abaabba
- abaabbb
- abbaaaa
- abbaaab
- abbaaba
- abbaabb
- abbabaa
- abbabab
- abbabba
- abbabbb
- abbbaaa
- abbbaab
- abbbaba
- abbbabb
- abbbbaa
- abbbbab
- abbbbba
- abbbbbb
- baaaaaa
- baaaaab
- baaaaba
- baaaabb
- baaabaa
- baaabab
- baaabba
- baaabbb
- baabaaa
- baabaab
- baababa
- baababb
- baabbaa
- baabbab
- baabbba
- baabbbb
- babaaaa
- babaaab
- babaaba
- babaabb
- bababaa
- bababab
- bababba
- bababbb
- babbaaa
- babbaab
- babbaba
- babbabb
- babbbaa
- babbbab
- babbbba
- babbbbb
- bbaaaaa
- bbaaaab
- bbaaaba
- bbaaabb
- bbaabaa
- bbaabab
- bbaabba
- bbaabbb
- bbbaaaa
- bbbaaab
- bbbaaba
- bbbaabb
- bbbabaa
- bbbabab
- bbbabba
- bbbabbb
- bbbbaaa
- bbbbaab
- bbbbaba
- bbbbabb
- bbbbbaa
- bbbbbab
- bbbbbba
- bbbbbbb