HyTime, the Hypermedia/Time-based Structuring Language (ISO/IEC 10744:1992), was developed in conjunction with the development of SMDL, the Standard Music Description Language (ISO/IEC CD 10743). Although space permits only the inventors and principal developers to be named here, several hundred others contributed to the work through meeting participation and written contributions over a ten-year period.
The first effort to combine descriptive markup, hypertext, and multimedia began in 1984 when Charles F. Goldfarb of IBM Research, the inventor of SGML, proposed an ANSI project to standardize music representation in SGML. He convened an ANSI study group on May 7-8, 1985, co-hosted by IBM and Stanford University's Center for Computer Research in Music and Acoustics (CCRMA), in which 32 people from the music and music software communities participated. They reviewed and endorsed the proposal (ANSI X3 SD3-542D, 1985/06/21), which describes several music and hypermedia applications:
After approval by ANSI (1985/11/22), the first meeting of what is now ANSI X3V1.8M took place in July, 1986, in Half Moon Bay, California; the meeting was hosted by Passport Designs. The committee was (and still is) chaired by Goldfarb, who invented SMDL and HyTime and was Project Editor and technical leader of the standards development effort in both ANSI and ISO. He was joined at that first meeting by Steven R. Newcomb, then with the Florida State University Center for Music Research, and now President of TechnoTeacher, Inc., who serves as Co-Project Editor and Vice-Chairman in ANSI. Goldfarb and Newcomb were the principal designers of both standards, although others made notable design contributions that are described in this history.
SMDL's architecture recognizes that music information, like other documentary information, is founded on certain abstractions (known collectively in SMDL as the "cantus"), from which one or more perceivable representations are derivable. In music, the perceivable representations can be visible "scores" (sheet music), gestural instructions for realtime control of musical instruments (as may be derived from MIDI sequences), recordings of musical performances, musicological and theoretical analyses, etc. Bernard Mont-Reynaud of CCRMA contributed significantly to this architecture.
A key accomplishment of SMDL was the separation of temporal information (the "time model") from other music information, and the time model's principled division into duration (the HyTime "extent" construct), meter (the SMDL "stress template" construct), and tempo (the HyTime "projection" construct). This work was chiefly due to Newcomb and to Alan Talbot, then of New England Digital Corp., with Steven Bertsche (University of Delaware), Garrett Bowles (UCSD and the Music Library Association), Philip Farrand (Opus Dei), Ron Gorow (Ron Gorow Music), Dorothy Gross (Control Data), Jerome Wenker (Unisys), and Fredric Wilson (Morgan Library) also contributing significantly. Neill Kipp (now of TechnoTeacher) completed the first practical demonstration of the time model as part of his Master's thesis in Computer Science in 1990.
Goldfarb added to the time model his concept of "dimension referencing", in which synchronization is expressed by representing the onset time and duration of a note or other event (its "dimension") as a function of the dimension of the event with which it is synchronized. Newcomb and Kipp made proposals that simplified dimension referencing while retaining its power. Later, during the development of HyTime, at the suggestion of Roger Price (IBM France) that absolute dates and times be supported, Goldfarb extended the time model to allow dimension referencing to the calendar.
Goldfarb and Newcomb created a pitch model that can represent any tuning system and any sets of interval and pitch classes, and that is capable of abstractly representing any equal tempered, just intoned, ethnomusicological, or arbitrary synthetic tuning system, with or without the concept of octave-by-octave repetition of pitch classes. Ron Gorow (Ron Gorow Music), a freelance Hollywood arranger and composer, invented a model for chords that distinguishes the abstract intervallic structure from the various notational representations used in popular, jazz, and country scores. Donald Sloan (then of SUNY Binghamton) and Gorow then extended the model to allow abstract representation of the information provided by traditional figured bass notation commonly used in Baroque music.
The requirements and existing practice for researchers, librarians, and musicologists were contributed by delegates of the Music Library Association, (principally Garrett Bowles but also Caroline Rabson, Richard Smiraglia, Lynne Toribara, and Phil Youngholm), together with Eleanor Selfridge-Field of the Center for Computer Asisted Research in the Humanities. These requirements included the need for arbitrary cross-references among parts of the cantus and its various perceivable representations. Link element types designed specifically for these purposes appeared in early 1987 drafts of the SMDL language design.
Other experts in music, music publishing, composition, performance, education, research, and music software and hardware systems also contributed time and advice to the SMDL work. Some of them, not mentioned elsewhere, are (in alphabetical order): Douglas Arndt, Geoff Brown, Wendy Carlos, Nancy Colton, Orion Crawford, Perry Devine, Eric Foxley, David Glasier, Robert Hall, Craig Harris, Wes Horlacher, Cleo Huggins, David Jaffe, Eitaro Kawaguchi, Douglas Keislar, David Kusek, Mark McGurty, Dave Oppenheim, Larry Polansky, Stephen Pope, Joseph Smerdel, and Leland Smith.
In order to represent music cross-references more elegantly and extensibly, and to represent the complex branching and repetitions that occur in musical structure, Goldfarb invented a general hyperlinking and location addressing architecture and documented it in the May, 1989, draft standard. Like the music information that inspired it, the architecture enforces the distinction between, on the one hand, identifying objects, and, on the other, using those identifications to express hyperlinks or other relationships among the identified objects. Together with Kipp and Steven J. DeRose (Electronic Book Technologies), he later designed HyQ as a query language interface to the addressing architecture.
The hyperlinking and location addressing architectures, together with the division of the time model into general and purely musical aspects, enabled a shift in the SMDL committee's focus from describing purely musical abstractions to generalizing those abstractions for other kinds of information. Len Bullard (Unisys) not only noted that SMDL, as the first standard to represent timing and synchronization abstractly, would be needed by the U.S. Department of Defense (a client of his employer), but later also predicted it would be used for modeling the behavior of complex systems, such as human organizations. Eric Johnson and Carlton Neville (consultants for the CIA's Foreign Broadcast Information Service) contributed ten scenarios for non-music, non-multimedia applications of SMDL, including modeling shuttle launches.
On June 19, 1989, a proposal ("X3V1.8M/89-25") was filed with ANSI to move the more general "time model" matters from SMDL to a separate standard, developed by the same committee, known as the "Hypermedia/Time-based Structuring Language (HyTime)". Thereafter, the SMDL draft was divided into parts to isolate the HyTime material, which was now identified by that name.
Dave Gunning (U.S. Air Force) and Bryan Caporlette (now of Passage Systems) presented the Air Force requirements for what later became the HyTime-based MIL-D-87269 standard for representing revisable databases for the support of Interactive Electronic Technical Manuals (IETMs). Caporlette also contributed the requirement for the "reftype" common attribute, a means of validating that the anchors of the multitude of hyperlinks in IETMs were of the correct element types.
Ted Nelson (Autodesk) gave an extended presentation to the committee outlining his approach to making hypertext widely available, which greatly influenced the committee's ideas on the scope and field of application of HyTime.
The HyTime design, though arrived at independently, layers anchor resolution and access in a manner similar to the Dexter Hypertext Reference Model, developed by a group under the technical leadership of Frank G. Halasz (Xerox PARC) and Mayer Schwartz (Tektronix Labs). The Dexter group was organized by Jan Walker and John Leggett, with the help and participation of Rob Akscyn, Doug Engelbart, Steve Feiner, Don McCracken, Norm Meyrowitz, Tim Oren, Amy Pearl, Catherine Plaisant, Randy Trigg, and Bill Wieland. Tim Oren (then of Apple Computer) prepared a report (X3V1.8M/90-54) comparing the fifth draft of HyTime (July 25, 1989) with the Dexter Model, noting that "the Dexter group generally agreed that the implicit HyTime data model is remarkably robust considering that it arose from one application scenario." It suggested some changes -- which were adopted -- and acknowledges that the HyTime distinction between linkends and anchors would be incorporated into the Dexter model. Oren (separately) also identified the requirement that led to the "spanloc" architectural form.
At a notable meeting in November, 1990, Brian Markey, then principal multimedia architect for DEC, alone among the 17 members in attendance, insisted that HyTime needed to represent space (graphics) as well as time (audio). Goldfarb resolved the issue by extending the time model to become the finite coordinate space (FCS), in which the application could define the number of axes and the unit of measure (temporal, physical, imaginary, or other) that applies to each.
Victor Riley (now of Silicon Graphics) contributed the Intermedia project notion of adding the user's traversal history to the hypermedia web. That idea, along with several dissimilar scenarios from industry, education, and the military, were eventually addressed by a common solution, activity tracking, which Goldfarb introduced at the February 1991 meeting.
During the development of HyTime, concepts under consideration were tested by the creation of detailed examples, and by prototype implementations. Vicky Newcomb and Neill Kipp (TechnoTeacher, Inc.) created examples and together with Anders Franzen (TechnoTeacher, Inc.) implemented prototypes.
The split of HyTime into a separate standard meant that two DTDs, rather than one, would be needed, with a complex interface between them requiring scores of parameter entities. To eliminate this problem, Goldfarb invented "SGML architectural forms" and, at the June, 1990, meeting, submitted a revised clause of HyTime that used them. The committee approved the idea and Goldfarb subsequently converted HyTime from a document type to an architecture.
SGML architectural forms, which are essentially superclasses from which element types can be derived, have since gained wide acceptance in the SGML community and are now being used to harmonize related DTDs in a rigorous and formal manner, regardless of whether the applications also employ HyTime architectural forms.
The first International Standard (beside HyTime itself) to use architectural forms is ISO/IEC 12083, an industry application standard for the representation of electronic documents, originally developed by the Association of American Publishers. The standard is edited by Eric van Herwijnen, who introduced HyTime-conforming element types for hyperlinking and location addressing.
In addition to conforming to the HyTime architecture, ISO/IEC 12083 also conforms to the SGML Disability Access (SDA) architecture, which allows virtually instantaneous production of Braille as an inexpensive by-product of document development. The use of SGML architectural forms to represent SDA is due to Yuri Rubinsky (president of SoftQuad, Inc.). SDA is the first standardized use of non-HyTime architectural forms.
HyTime and SMDL were proposed as International Standards in January, 1991. This action brought the participation of international standardization experts, including Yushi Komachi (Matsushita, Japan), Hugh Tucker (Documenta Aps, Denmark), and Martin Bryan (The SGML Centre, UK).
At international committee meetings, Rita Brennan (Apple Computer, USA) presented a multimedia interchange format called BENTO, designed by Jed Harris (Apple Computer). With Harris, Goldfarb created Standard BENTO (SBENTO), which applies the key ideas of BENTO in a HyTime/SGML context. Lloyd Rutledge (University of Massachusetts at Lowell) made important suggestions for improving the description of HyTime's dimension referencing facilities.
Carsten Bormann and Ute Bormann (Technical University of Berlin) contributed a detailed critique of HyTime requesting, among other things, a more formalized specification of the language. This request resulted in the development of HyLex, which is both a facility of HyTime and the means used to document its lexical constraints.
DeRose and David Durand, on behalf of the Text Encoding Initiative (TEI), contributed an exhaustive review of HyTime with respect to TEI requirements for the representation of scholarly documents. These included the need for lexicographic ordering, improved treatment of document sets for distributed multimedia, non-HyTime traversals, and more general and compact location forms and queries. DeRose also contributed a design for one-to-many and many-to-one lexicographic ordering.
HyTime was published as a Draft International Standard on October 10, 1991. It was balloted internationally for six months, until April 10, 1992, and was approved by all 23 ISO national bodies and liaisons who voted.
HyTime became an ISO and International Electrotechnical Commission (IEC) joint standard upon publication as ISO/IEC 10744 on November 1, 1992.