| NIST'S PLANS FOR A FLYING LEAP, BACKWARDS! |
Michael Gorman
Whitemarsh Information Systems
Just when I thought that Federal FY-97 (October 1996-September 1997) was going to be another banner year for high quality database consulting, my understanding of plans under development at the National Institute of Standards and Technology (NIST) have thoroughly ruined my outlook. Before I tell you about my understanding of these FY-97 plans, I must back up to a bygone era of "waste-of-time" consulting that NIST's previous management had obviated because they truly understood the data management business. This "old" management however has been retired or "redirected."
I'm in the data management business with an emphasis on database. Database is a computer related business that brings order, structure, single source and sharing the critical business data. I was part of database's beginnings in the late 1960s. From then until the late 1980s, most database management systems (DBMSs), the computer software products that define database structures, store, update, report, and protect business data were unique to each vendor. It was like buying a Ford and then finding out that you must use Ford gas, Ford tires, Ford roads, Ford oil, and Ford every thing else. Where would we be if GM owned the North-South roads and Ford owned the East-West roads. Silly? Unthinkable? Or, just plain stupid?
Every DBMS vendor, like IBM, had different systems. Because they looked and acted differently, business applications that used one DBMS couldn't be used with another. Government agencies, once they selected a DBMS were locked in for years and years and years.
While the cost to buy a DBMS was small (less than $200,000), each significant DBMS application cost ten times more, or $2,000,000. If an agency had 50 DBMS based applications they had a $100 million sunk cost. Most, if not all that $100 million had to be trashed and re-spent when the DBMS was changed!
Because the cost of change was so expensive, agencies spent $100 to $300 thousand to pick the "right" DBMS. The rationale for the analysis was sound. Buying the "wrong" DBMS was frightfully expensive. The consulting opportunities for DBMS experts like me were every where and were three types: helping to create 20 to 30 page questionnaires, helping to write 100 to 300 page questionnaire responses, and finally spending from weeks to months helping agencies evaluate vendor responses. One agency even paid me to develop and present a course that taught them what their procurement questionnaire meant! All this DBMS and vendor specific work was necessary because vendor features were closely guarded, proprietary, highly copyrighted, and subject to protracted suits if any vendor attempted to implement another vendor's features or facilities.
The DBMS standards work began in the late 1960s with the Committee on Data Systems and Languages (CODASYL). CODASYL, during the 1968-1978 time frame, produced detailed standards specifications, but since there were no conformance tests and certifications, these documents merely became dust collectors as vendors interpreted, changed, and extended them to suit their own liking. In 1978 ANSI took its turn and by 1980 one standard, the Network Data Language (NDL), was presumed ready. Phil Shaw of IBM, however, put it best when he said, "having only a standard data definition language without also having a standard data manipulation language is like having talk with no walk." So, ANSI continued its efforts, and because of a NIST linguistic invention had a complete NDL standard by 1984. In 1982, ANSI also started to standardize another model of data, the relational model. The resultant standard was SQL.
SQL was not however destined for the dusty book shelves. IBM, Oracle, and then Digital were fully committed to deliver SQL based DBMSs. But would these DBMSs really act the same way? CODASYL DBMSs didn't. NIST again put forth the solution: conformance tests and product certifications. The model NIST uses costs the Government nothing because 100% of the NIST costs are recovered from the vendors who want conformance certificates. Through these tests, NIST ensures that the vendor based SQL DBMSs behave exactly the same way. That means that users can employ SQL DBMSs with minimum risk.
From 1986 on, and specifically because of ANSI, DBMS features are specified in common by vendors, government agencies, and users. These new features are proposed without the normal non-disclosure, proprietary forms. Features are freely discussed, debated, and then adopted through democratic votes. Once features are adopted, they are available to all to implement because everything standardized by ANSI becomes "public domain." How are DBMSs distinguished in the marketplace? Simple, price and performance. But, not on vendor unique features that locked customers to vendors.
Because of ANSI/SQL and because of NIST's conformance and certifications, all this DBMS and vendor specific selection and evaluation consulting started to disappear by the end of 1986. Through the Federal Information Processing Standards (FIPS), NIST told Agencies that they could only procure "NIST approved" DBMSs. Overnight, almost all vendor proprietary DBMSs disappeared.
The "teeth" in the NIST's dictum to only buy "NIST approved" DBMSs is achieved through NIST's extensive suite of conformance tests that force DBMS vendors to "walk" their "talk." If vendors want to sell Agencies their DBMSs they have to show their NIST SQL conformance certificates. NIST even caused ANSI to include a FIPS "flagger" in SQL that identifies vendor unique statements! Overnight, DBMS changed from a speciality, Agency locking business to a highly competitive, commodity business.
In addition to making procurements faster, easier and cheaper, ANSI/SQL databases are able to be exchanged, Agency to Agency without extensive data conversions. The same computer programs can be used with different ANSI/SQL DBMSs. Universities teach ANSI/SQL to students who use these skills for different companies and Agencies. These benefits are great for Agency computer programs and data, and their ability to use commonly trained technical staff.
Because I am no longer focused on working all three sides of the DBMS procurement business, I concentrate on really helping agencies advance their data management skills and develop sophisticated agency wide, heterogenous hardware and operating system environments.
Because of ANSI/SQL standards, data and its meanings can be exchanged with and between Federal agencies, between Federal, state and local agencies, between medical institutions, courts, universities, businesses and industries.
In the years since NIST enabled DBMS features to be public domain, SQL DBMS vendors sprang up like spring flowers. Previously, no business would dare risk their very existence on a vendor locking DBMS. Because of NIST's conformance testing and certifications, businesses can try a DBMS, stick with it, or leave. The only real cost is a database unload and reload into the next ANSI/SQL standard DBMS. This $1000 cost is in stark contrast to junking $2 million in software! Real benefits have arrived.
Now, however, from my understanding of NIST's FY-97 plans they only intend to complete part of the SQL tests, stop all SQL/DBMS product certifications, and terminate test development FY-97 and beyond.
One could assert that since SQL is a 10 year old standard it must be mature, old technology, and doesn't need more SQL tests or product certifications. The response to all these assertions is a resounding NOT TRUE. Most of SQL's really valuable features are just now being developed. Completely untested will be SQL's most value added extensions that deal with sophisticated new types of data, its full SQL programming language, its just standardized call-level interface to traditional programming languages (COBOL, C, and Ada), and yet to be completed features addressing full text, multi-media, engineering and science.
Stating that SQL testing is now complete is like stating that an aircraft carrier is complete just because it has enough keel and structure to float. Floating cannot possibly be THE valid test, only full functionality and usefulness can be.
Because of NIST's FY-97 and beyond plans, SQL's conformance tests and certifications, that is, those beyond the SQL shell will be left to the ANSI/SQL vendors. They however have no motivation whatsoever to perform full and complete testing nor self policing. Only the largest buyer has that motivation, and in the case of ANSI/SQL the largest buyer is the United States Government.
Simply put, without robust test development and conformance testing by NIST, DBMS will return to the days of vendor specific, conflicting features and facilities that will lock Federal agencies into one vendor, or make DBMS frightfully expensive to acquire, use, and dislodge.
If vendors did believe in level playing fields, highly interchangeable databases and application programs, they would all be protesting NIST's abandonment of ANSI/SQL certification and conformance testing. NIST's conformance tests and certifications leveled the playing field. Is it mere coincidence that the two biggest DBMS vendors (75% market share) are silently sitting on the side lines? Could they just be biding their time till NIST dismantles its ANSI/SQL certification and conformance testing program before bringing their vendor specific and unique features to market?
Once these new vendor-locking features are deployed through aggressive marketing campaigns, Agencies will be unable to know they are unique because NIST won't have the conformance tests and certifications for these newly developed features. Agencies will then either be condemned to the unsophisticated data management of the 1980s or be sucked into using the new sophisticated, but vendor locking data management features of the future. No longer will the playing field be level. Rather, only the few very big vendors will remain. Competition will drop, prices will rise, and portability of programs, data and trained staff will end.
For agencies to have any chance to keep control of their data management futures, they must either develop their own cadre of data management experts (build their own in-house NISTs) or turn to computer experts like me. As an industry insider, I know how to build the critical questionnaires, and help agencies do evaluations. Now, because of NIST, I must stop my work advancing the practice of high quality, enterprise wide database and return to the highly lucrative, but completely wasteful practice of doing useless, unnecessary DBMS features selection and evaluation work.
Because of NIST's plans, agencies must put their plans for easily understood and exchangeable scientific databases on hold. Because of NIST's plans, the critical underpinnings for the new U.S. Weather modernization program (hundreds of millions of dollars already spent) will be ripped away. Because of NIST's plans, the technology basis for the replacement system for the Army Reserve System will be crippled.
No longer will DBMSs be commodities. Slower, more difficult and more expensive procurements, along with data incompatibilities, crippled project missions will return.
Even though I can now dust off my special DBMS courses, can count on contracts for data conversions and complete program rewrites, I find such work empty because it only addresses evaluations, conversions, and program rewrites where none are now needed. The real work that satisfies me is using sophisticated DBMS features that were defined through standards, available to all to implement because of their ANSI public domain characteristic, and were guaranteed to work the same way through NIST conformance and certification. Because of NIST's plans, all this quality work will soon be gone. In its place will be hundreds of consultants earning $100 thousand or more doing the useless work "old" NIST had previously obviated. And, by my calculations, "new" NIST's projected annual savings of $700,000 will be replaced by annual Agency consulting contracts costing upwards to $20 million! If that's efficiency in government, thanks, but no thanks.
Mike Gorman is an expert on the content of this article because he has been involved in database activities for almost thirty years. During that time Mr. Gorman has been a database teacher, vendor, an evaluator, designer/developer, and consultant to both Government and industry. Mr. Gorman has taught and lectured in Graduate schools. During this time frame he also placed data management software and designed and developed database applications for over 35 Federal Agencies. Mr. Gorman has consulted with and been lead designer on a number of critical database applications for US and world wide corporations. He also published four text books, and finally, is an officer of X3H2, the Database Languages Committee which standardizes the ANSI/SQL language. Mr. Gorman has participated in X3H2's data management standards work for the past 18 years.
