Are Data Dictionaries like English Dictionaries?
Dec 2, 2024
Enterprise data dictionaries should be child’s play for LLMs. After all, most systems have <5,000 fields of which <500 are commonly used, while English has a hundred times as many (~500,000 words of which we use ~5,000). Unfortunately, the task is not so simple. The Oxford English Dictionary would not be much use if we didn’t inherently know how to use words-if the “deep structure” of language were not embedded in our brains. The functional equivalent for enterprises-their “logical data model”-has never been successfully established. The closest analog-Master Data Management-has had mixed success. LLMs would chew through this problem in a heartbeat if they were given access to data models and data from all systems in the way they have had free reign over English text on the web. Alas, this is never going to happen. Using LLMs to catapult Logical Data Management will require a different approach.