Ever since OpenAI launched ChatGPT last November, it has been all the rage. It has attracted enormous attention in the media including from some of the leading thinkers in the world. Stakeholders are experimenting with all types of new use-cases (still mostly language-, image-, and video oriented use-cases) and are both fascinated and scared at the same time. In this article, we dive deeper into what ChatGPT is and how it works, why it is a game changer, and what it means for our clients in the traditional industries that make physical products.
ChatGPT 101: How it works
ChatGPT (also referred to as GPT-3) is a giant neural network model, more specifically a large language model (LLM), consisting of nearly 175 billion neural network weights divided among an input layer, an output later, and about 100 interior layers. The model has been trained using text from billions of web pages and books. The reason that there is so much hype around ChatGPT is because the program can have natural human-like conversations while keeping the context of the entire conversation in mind. Trained using billions of facts — true or otherwise — the program can answer questions as diverse as “Tell me about the solar system,” “Briefly tell me about politics in Malaysia,” or “List the different forms of art” as well as address follow-on questions such as “Which is the most popular form?” while keeping the context in “mind.” The responses from the program are nearly indistinguishable from how a human being would respond. In some situations, the program even creates new words and phrases that are not to be found in dictionaries; when I asked ChatGPT to create a new word, it came up with “Nuventive — Describing someone or something that is both innovative and intuitive.” For now, ChatGPT’s creativity is mostly limited to language areas; other programs such as Dall-E are doing something similar with images. Before we see how all of this could be relevant to manufacturing industries, I want to take you all on a journey spanning ancient India, neuroscience, creativity, and AI.
The devil lies in ignoring the details
How did ChatGPT become so creative? To answer that question, we need to understand the fundamentals of human creativity, which we will do using the ancient Indian parable of “The six blind men and the elephant.” The story goes that a group of blind men, who have never known what an elephant is, happen upon an elephant in the forest. They each touch a different part of the elephant — the trunk, the leg, the ear, the tail, the body, and the tusk — and describe the elephant as being a snake, a pillar, a fan, a rope, a wall, and a spear, respectively. And therein lies the core idea of creativity.
Studies indicate that the human brain’s storage capacity is limited. With 125 trillion synapses, each of which can store approximately 4.7 bits of information, an average human brain can store only around a terabyte of data. Compare that to the estimated amount of data on the internet — 64 zettabytes — 9 orders of magnitude more than what the human brain can store. And we haven’t even considered data from other sources such as books, reports, businesses, and government. Clearly, the human brain does not have the ability to store and process all of the data out there. But this Achilles’ heel is also one of the critical drivers behind human creativity. Because we do not have the ability to store large amounts of data, we resort to reductionism, that is, capturing millions of dimensions of data into a few key concepts such as the Newton’s laws of motion or Einstein’s special theory of relativity. Going back to our elephant example, reductionism would mean that the human brain, rather than remembering every minor contour of the elephant, abstracts the elephant into a collection of a few key features — legs, large fan-like ears, a long trunk, and tusks. In other words, by applying reductionism we automatically become like the six blind men — only partially knowledgeable but also much more imaginative.
This is the secret to ChatGPT. Traditionally, most AI programs have been trained on massive amounts of data with the ultimate goal of predicting the final result (in the case of our example, this means describing the elephant in excruciating detail); the focus is on the final output layer while the interior layers are largely ignored in favor of that final result. ChatGPT or GPT-3, on the other hand, stops at one of the interior layers just before the output layer. These interior layers tend to capture the features of the problem (in the elephant example, features such as fan-like ears, trunk, tusks, etc.). With reduced information, ChatGPT now is forced to extrapolate and be creative like the human brain1, 2.
Applications of ChatGPT in manufacturing
As mentioned earlier, ChatGPT is a LLM, and as expected, some of the first applications that will be unleashed will be in the area of language — Microsoft Office applications (Copilot), finding relevant information from internal company knowledge databases, and generating reports. However, we can use the concept of reductionism to extrapolate other uses for ChatGPT. Generally, ChatGPT is predominantly good for tackling large system-level situations that involve near-unlimited combinations of smaller components. Language is one of those system-level situations. Language is really a complex mode of communication that involves bringing together words, phrases, grammar, and punctuation to deliver some meaning. Such system-level situations are very common in traditional industries such as manufacturing, energy, chemicals, and utilities, suggesting that ChatGPT is likely to play a critical role in this space. Some of these larger system-level situations in manufacturing industries where generative AI’s creativity will be valuable include:
- Designing complex system-type products via generative design or informatics. While some of this is already being done, they are still being applied at the component level or to simple systems. ChatGPT-type algorithms can help innovate at a system level (like redesigning a whole car or a transportation system).
- Applying six sigma to identify innovative opportunities to eliminate bottlenecks in complicated manufacturing processes.
- Using operational analytics to identify novel efficiency improvement opportunities in complex manufacturing operations.
- Making strategic business decisions while considering multiple dimensions such as macroeconomics, competition, supply pressures, consumer behavior, etc.
It should be noted that generative AI technologies are still somewhat unstable, meaning that human oversight will still be needed in most of these applications. Moreover, clients should not expect generative AI algorithms to come up with paradigm-shifting ideas any time soon.
Recommendations
- ChatGPT as is can be used only for language-type applications noted earlier. For other applications, clients will likely have to build and train their own generative AI platforms.
- The publicly available version of ChatGPT was trained using text from billions of web pages and online books. Clients in the manufacturing, energy, and chemicals sectors are unlikely to find that much data to train similar algorithms. They will have to wait for smaller, generative, neural net algorithms and leverage other techniques such as transfer learning to train these algorithms.
- It is worth noting that ChatGPT uses 175 billion parameters and is currently restricted to run on the cloud (GPT-4 has 100 trillion parameters). In other words, the algorithms are not likely to offer any real-time insights. Clients should prioritize applications where delays will be acceptable.
- ChatGPT is not going to be the one and only algorithm around: Its strength is its creativity. Most tasks in our daily lives or on the manufacturing floor are mundane and involve little creativity. What this means is that traditional AI algorithms are likely to exist alongside ChatGPT-type algorithms.
Despite all these challenges, clients should note that ChatGPT is definitely a game changer in the realm of AI and will likely unleash a plethora of applications in multiple industries. Clients should track developments closely.