A lot of use of LLM’s is around research & debugging. I write code by hand and occasionally let it auto-complete, but beyond that, most of the LLM work is re-learning some of the fundamentals, or asking it to help with compiler & pipeline errors. This got me thinking about what thought leaders in the industry shaped me over the years, and how the Internet’s distilled knowledge in these LLM’s is shaping the programmers of tomorrow.
Melting Pot, Java, Object Oriented Programming, Functional & Extreme Programmers
My career has been shaped by 4 distinct stages in terms of the community of thought leaders that heavily influenced me and my career trajectory.
In my early career it was the melting pot of web developers who poured into Flash and Flex. After that community started to mature and spread into all types of businesses, it became heavily influenced by the Java and Object Oriented Programming communities with a slight tinge of .NET. I started to stumble more and more onto various Functional Programming influences in both blogs and conference talks, and later on type theory. During my consulting stints I started meeting some Agile & Xtreme Programmers who helped fill in the technical practices needed for Continuous Delivery that all the Agile people I worked with never seemed to know.
I can name distinct individuals, blogs, conferences and talks, as well as specific libraries that helped create that set of learning channels that I followed over time.
LLM Knowledge Soup
When I ask the LLM questions, sometimes I can catch a whiff if someone or something I know being distilled in the response. For example, if I ask it early 2000’s OOP questions, I can sometimes see it’s pulling from Martin Fowler’s blog, or some other similiar blogs I’ve read in the past.
In more modern topics like type theory, type usage, and TypeScript, it’s sometimes clear it’s read 1 long book or article as it has an agenda or stance that is a distinct style. Other times, it might be more hard to pin down BUT it’s clear it’s reading past chats, like ChatGPT/Codex does in the browser, so it’s adding those references in. That means not only is it biased by multiple blogs & YouTube talks that I’m not sure the source, but also my own past biases in the questions. So the responses are biased by things I forgot I even put into past prompts.
I’ve been doing this for about 2 years now; once Claude Opus 3 and 3.5 came out, clearly both it and ChatGPT, and later Codex, got some great training data enhanced to understand how my programming prompts/questions related and made it great for research and help in training; meaning, distilled knowledge examples I could use to help teach others.
My concern is I can’t always have it cite sources. Because I’m familiar with the topics I’m researching, it’s much easier to see when it’s hallucinating or gaslighting me, but other times when I explicitly asked, it didn’t have any.
So these LLM’s that have trained on the entire internet of tech blogs and tech video talks, but it’s so mixed you can’t tell where this knowledge came from. Generally I’m not concerned if juniors read some of these responses, but I am concerned it’s just one viewpoint; meaning “just Claude Sonnet 4.6” or “Just the latest Codex”.
Mixed Viewpoints
Programming has always been interesting to me because you’ll have these “brilliant engineers” and get 2 of them in a room, and they can’t agree on fundamentals. If you know the fundamentals and listen to the debate, you can see how “both are correct in various contexts”. You then have to do your own internal thought, soul searching, and leaning on your own experiences WHILE ALSO trying to keep an open mind there may be nuances your missing.
For example, hearing Uncle Bob vs Kent Beck & Martin Fowler talk about the two approaches to Test Driven Development using the London Style vs. the Chicago Style. They’re distinct enough that you can get a sense of both styles, their pro’s and con’s, and how you’d apply to work and in what situations. The key here is Uncle Bob, Kent Beck, and Martin Fowler are all distinct individuals. Each has their own bias, style, and way of communicating. Bob & Kent + Martin at the time did not agree; yet most people in the OOP programming sphere agreed with their views. So you could specifically cite all 3 speakers/writers in your own follow up conversations with your team/colleagues and compare/contrast their viewpoints. This includes the fact that while Kent & Martin agree, they have different ways of communicating that agreement; e.g. different ways of phrasing it and different code examples.
LLM’s? You specifically have to prompt them to role play, meaning “Please respond with both views of how Uncle Bob would approach this problem vs. how Kent Beck would”. By default, they answer you. At the time of this writing, Claude Sonnet/Opus will inject more sycophantic and/or agenda styles to the responses whereas Codex will be more “just the facts”.
Nuance and Differing Viewpoints After the Fact
Now, I’m middle age. I know, and have experienced, many Juniors who’ve never heard of Uncle Bob, Martin Fowler, Kent Beck, Dave Farley, Jon Van Neumann, Dr Nicole Forsgren, Scott Wlaschin, etc. However, I can give them evergreen blog posts, links to books to purchase, and YouTube talk links of these thought leaders for them to learn them, they’re viewpoints, and the topics they’re covering.
For those primarily using these AI tools to learn nowadays vs. the Googling of the past, I’m wondering what the long term affects of this style of learning will be. Meaning “You ask the AI first” instead of Googling it, and reading a few blogs or watching a few YouTube programming or conference talks. In the past, we’d get a series of viewpoints, sometimes groupthink or cargo culting, no doubt, but at least it was spoken differently with each person’s unique perspective.
Now the default for those using these LLM’s to ask their questions is a single viewpoint of the mashed together training data, with only the nuances brought up if the prompt/rules indicate to do so.
Conclusions and Concerns
Is the viewpoint too narrow? Is there truly going to be a loss of nuances? Is the training set overly biased? Do the prompts truly determine someone’s belief systems going forward? I noticed Gemini and other LLM’s are often hallucinating, but now that this is the primarily search result on Google, do new developers know this? What do they do when they’re not familiar with the subject matter and they don’t know if the AI is hallucinating the answer?
I’ve taught my kids to be aware of the above issues, as well as my fellow devs, but this is worldwide normalized now.
I’ll continue to blast the old and new blog posts, books, and video conference talks I know are great and helpful to learn to those I learn & teach with, but I do have concerns about the questions above now that these tools are so pervasive, and often the default, for many people to learn.
Leave a Reply