DBDS - Publications - Newsletter:The Metadata Mystique

Active Metadata

Adrienne Tannenbaum

In today's business climate, information is probably the biggest part of our professional lives. Deciding whether or not to reduce our prices, deciding whether to hire additional employees, deciding if an ongoing project is going to remain profitable. The small company relies upon its built-upon experiences in order to make these decisions. The large corporation needs to rely upon the experiences of many, some of whom may not in fact be a part of the available knowledge base - hence the reliance upon data. When we have data without the knowledge base, our biggest asset is often the accuracy of the associated metadata.

Metadata - Right or Wrong? Good or Bad?

In the good old days, there was no metadata. That's right..and there was no need for any since everything was well understood, and if not, everyone knew how to find out the missing facts. Today metadata is an integral part of our daily activities. So much so that we make a point of creating and updating our metadata world on an almost daily basis. Many of us may think that we are creating new sources of data, but in fact, each source of data automatically creates and demands associated metadata. All of those instantly created spreadsheets, documents, diagrams, and reports are immediately in need of supporting metadata - all that is necessary to locate, identify, understand, source, and access our new creation after it is completed.

Depending on the software that is used to create our data (structured as well as unstructured), some of the necessary metadata is created automatically (unbeknownst to us in many cases) and perfectly. Consider the fact that once we create a spreadsheet (using Microsoft Excel, e.g.), we will always be able to access that spreadsheet - based of course upon the fact that the automatically generated file type (.xls) will always invoke the procedure required to open the spreadsheet and display it. But other metadata specifics related to a spreadsheet may not be as automatic and may not give us the same results. A filename, required when we want to save a spreadsheet, has as much value as we ourselves place in its inherent meaning. A filename of spreadsheet.xls for example, tells us nothing other than the fact that the contents of that file are an Excel spreadsheet. The identification and location of this file are then not as intuitively obvious to us unless we place these objectives in the filename itself. Even good filenames only show us the creator's perspective on what remains to be seen inside.

But most of us are more concerned with what is actually inside of these data stores. Even though we go through enough trouble trying to locate the data, we clearly have to know the boundaries under which the data was created - for what purpose, why, when, where did it originate, who created it, who uses it….I could go on and on….few of us will debate the need for metadata.

Because metadata is such a crucial part of our lives, we create it. I write notes, update calendars, send emails, create documents, organize folders and files, and always make sure that the important parts of all of these activities are documented so that I can re-visit their origin. Good metadata - or bad metadata? Only time will tell.

The quality of metadata is something that is very dependent upon its creation and the surrounding circumstances. But even good metadata can become bad, over time. The optimal usefulness of metadata requires the following features:

· Metadata/Data Connection - when that which is being described changes, then the description must change instantaneously

· Metadata Availability and Access - if we have the greatest metadata in the world, but can't find it, or don't know how to look for it, or can't open or read it once we find it, well, what's the point?

These are the reasons for Active Metadata. Without that direct link, that direct connection, and a consistent ability to always locate all ends of the connection, our metadata is passive. Passive metadata requires a human update. Active metadata is always as up to date as it can ever be. Metadata is active when it represents that which it clarifies. Examples of active metadata include:

· URLs - always taking us to the spot that they represent. When a URL changes, it takes us to a different spot. We can run into trouble when we create a website, then passively document the associated URL in another place. When human intervention is required, even the most active metadata can become passive, or "out of sync".

· DBMS Catalogs - There is no way that a new column can be added to a table without changing the DBMS Catalog entry. That is what Database Definition Language (DDL) is all about

· Tables of Contents - In most documents, the Table of Contents immediately reflects page number changes, additional headings, etc. If human intervention is the only way to update page numbers, then the TOC can easily become passively unreliable.

When metadata is in demand, it is created. When bad or incomplete metadata is created, the demand remains. When complete, accurate, and good metadata is created, it will only remain that way if it is always accurate. Active metadata is always accurate because it is directly focused and dependent upon that which it describes.

The implemented metadata solution is the carrier and pointer to your needed metadata. If the solution takes you to inaccurate or incomplete metadata it is most likely being passively maintained. If the solution stores all of available metadata within its internal architecture, it could potentially become out-of-sync without automatic updates. Consider the investment in "automatic" and "active" links as part of your overall strategy.

Your customers are demanding for metadata. But they are demanding accurate metadata…and only active metadata is always accurate.

Adrienne Tannenbaum is President of Database Design Solutions, Inc, www.dbdsolutions.com a consulting firm focusing on the data issues of large organizations. She is a recognized metadata authority, having authored two well-received books. The most recent, Metadata Solutions (2002, Addison Wesley), discusses the world of practical metadata solutions in a complicated world. Adrienne lectures and teaches extensively both nationally and internationally.