Construction of Interface for Interactive Multimedia
Ravi... IDC Home Faculty at IDC	by Ravi Poovaiah, Professor, Industrial Design Centre, IIT Bombay       Abstract 1.0 Introduction 2.0 Presentation Level 3.0 Interaction level 4.0 Linking level 5.0 Organisation level 6.0 Variety of Information       References Abstract This paper looks at construction of interactive multimedia applications from the perspective of the communications designer, and tries to focus on issues that are of significance for a media-based communicator. There are many characteristics that are unique to this new media and these seem to offer creative opportunities to communicate with the media in radically different ways. To identify these, the construction of the interactive multimedia is conceived in discreet levels and the characteristics of each level explored for the potential choices it offers. It becomes obvious,therefore, that in order to gain proficiency in using this media, the design process in the construction of an interface for interactive multimedia needs to be conceived as a physical activity in terms of an interaction using our sensory capabilities; as a relational process that offers choices in terms of selection of information; and as an organisation of its various elements in terms of an arrangement across virtual space. This calls for new skills in understanding our ability to communicate in various sensory modes; of providing the user with various options; and of being able to visualise organisation of information in virtual space. It might even be essential to see these factors as relevant pedagogical inputs for those concerned with learning to work with this media. 1.0 Introduction Communication designers(or writers, directors, photographers, illustrators, etc.) have proficiency in using one or more variety of information, that are available to them in the form of text, audio, still images or moving images. What makes the interactive multimedia quite distinct from the other medias is its flexibility for integrating textual, aural and graphical modes of information; with provisions for a structure for linked elements through which the user can navigate; and for providing the ability to interact with information. If we were to consider interactive multimedia as a media for communicating information, then what are the choices that this new media throws up for the designer? And what specific skills are essential towards the construction of interactive multimedia? To understand this and to focus our attention, the construction of an interface for interactive multimedia is being conceived as having the following five levels: 1. Presentation level 2. Interaction level 3. Linking level 4. Organisation level 5. Sourcing level 1. Presentation level 'The restriction of the page, the frame, the aspect ratio of the TV set, the physical space of the exhibition hall and the manufacturing tools, all define the degree to which the audience or the user could interact with the medium'(Cooper, 1989). The pragmatics of the media have also defined their role as being suitable for interaction within personal space (such as a book), in social space (such as on television) or in public space (such as in a movie theatre)(Hall, 1966). This in turn has influenced the characteristics of the media itself and defined the suitablity of information for the particular media. 1.1 The display monitor Developments in technology has influenced the devices that are required for the presentation of information available in the form of text, audio or images. For example, the video monitor (the most commonly-used display device for interactive multimedia) becomes the physical interface for interacting with the message (or information). The use of information from other medias has to be seen in the context of the monitor's potentialities and limitations - its physical dimensions, resolution, viewing distance, portability etc. Presently, because of its hardware configuration interactive multimedia seems suitable as a indivisualised interactive system. The video monitor appears to be a common solution to all kinds of situations. The potentials of multimedia presentation devices for the future seem to suggest that display devices can be expected to be quite flexible in terms of its resolution, along with variable aspect ratios and may even be suited for each of the above mentioned spaces. 2.0 Interaction level Information processors being an integral part of multimedia technology offers us a variety of sophisticated means with which to interact with a product. With the basis for transformation being digitisation, it can now make for easier conversion, inputing, transmission, receival and manipulation of inputs from the different media. The communication of information from and about these interfaces should not be misleading or confusing; the message should be revealed as fast as possible i.e., the information must be communicated properly, efficiently and conveniently (Poovaiah, 1991). The aim is to make this interaction easy, simple, convenient, familiar and friendly - all these, of course, being easier said than done. 2.1 Humanisation of product environment 'Sometimes these interactive products are capable of modifying their behaviour on the basis of some external variable. This means that they are no more passive, but they set up a relationship and they establish an interaction with the user' (Manzini, 1988). The product and the user act with and react to each other, and it is entirely possible that man-made objects begin to operate like they were alive for users. This means that products can be designed to have the ability to understand and accept changes' (Sunaga, 1993). All this can be thought of as a humanisation of the product environment that we make use of in our daily interaction. 2.2 Transparancy in interaction Following from the above is the contention that whenever people learn something sufficiently well, they cease to be aware of it and are able to focus beyond them onto new goals. Examples of this being the acts of driving, reading, writing etc. 'The most profound of technologies are the ones that disappear. They weave themselves into the fabric of everyday life until they are indistinguishable from it'(Weiser, 1992). Hence we need to be concerned with the method and quality of this interaction and ways to make this interaction as transparent as possible and not focus our attention on the devices that are required during such interactions. 2.3 Multisensory interaction It will also be possible to make use of many of our sensory perceptions very much in the manner in which we interact with the environment. Lets us pause for a moment and try this out an exercise: try to express yourself or introduce yourself in the following manner: a) by means of actions without using any words. b) by means of your ability to draw without using any text. c) by means of sounds that you can produce without actually using any words. Difficult as the task may seem in the beginning, one would be surprised by the amount of success that can be achieved in the above tasks (Kolli, 1993). This means that we have the ability to abstract messages and to translate these into one or other modes of representations that are used by the different senses. In our daily interaction with others we communicate by using many of our senses, such as words, gestures, or even our sense of touch. Since there lies the possibility of one being able to interact with multimedia by a choice of touching, gesticulating, speaking, pointing, writing, typing etc., designers involved with interactive multimedia have, therefore, to enhance the quality of interaction with the product. It will be necessary to develop the language that is best suited to communicate with each of these different sensory modes. 2.4 Quality of interaction Interactive multimedia seems to offer the means for the user to control the responses of the information transfer process, according to his pace and choice so as to dynamically interact with the knowledge base. User can interrupt, interogate, repeat or pause, adjust the pace, scan, review, locate and modify the material according their individual needs. Multimedia involves the user directly in a dialectical learning process emphasising the importance of active responses over passive reception (Schank, 1994). Interactivity provides the user with a means to interact with the medium in a meaningful and rewarding way so as to enhance information transfer. The same media that is able to provide flexible access to information can also encourage interaction. The process of interacting with a multimedia application can be made into an exciting, memorable and beautiful experience. Even the process of learning can be conceived of being an enriching one. It is possible to make the interaction process interesting, curious, stimulative and pleasurable by incorporating devices of puzzles, surprise, challenges, queries, evaluation and reward. Excitement and entertainment seem to be some of the sustaining factors for these kinds of interactions. Can some of these 'sensexciting' factors be made part of the environment of the new media? For example, these are the factors that seem to provide some of the essential features that go into structuring computer games. Educational applications in interactive multimedia could be quite entertaining. With many of these options, it is the designers task to make the interaction process a unique experience. 2.5 Interaction design In an interactive environment, the user interacts with the product, which in its own turn communicates with the user in some way or the other. This interaction should lead to an understanding of the product(Bonsiepe, 1993). The aim is to make the interactive multimedia environment respond in ways that are easier to grasp and understand. Interaction design would concern itself with the design of devices for operating a given multimedia environment; devices which convey information-feedback on the status of the application; the path or procedure to be followed in interacting with a given information; and the language involved in this interaction. To make it an efficient interaction, one needs to cross-relate the time required against the physical actions performed to operate a function. 2.6 Interaction activities If we look at the kind of functions that are essential for interaction, they seem to be those of being able to select information, of being able to modify the information and be able to respond in some form of language(such as voice or text input) . 2.7 Interaction devices The means of interacting with information was at first through the use of text commands. These commands had to be recalled from memory when required for use. Hence providing text commands as pull down menus or in pop up windows seemed logical as these did not require memorising. Later on text was being presented as buttons so that these would graphically denote their role as selectable items. Further developments included replacement of text commands with graphical images -popularly called icons. Use of an image to represent an activity or function seemed to have had the advantage of being easy to identify, to remember and to recall (though not all functions could be converted into icons and also, sometimes, graphical representations could remain ambigious and connote more than one meaning). To make the interaction interesting, the icons could be animated or coupled along with sound so that it became active and responded appropriately when a command was executed. Although they have made the interactive environment friendly to operate, these devices suffered from the drawback of making the user focus attention on them, rather than the media. For an interactive multimedia environment the focus, however, needs to be on the media and not on the devices that one interacts with. In my opinion, the solution seems to lie in the use of the displayed information itself as an active field for interacting and for providing the repeatedly required navigational devices outside the display area (i.e., in the mouse or as a pointer). 3.0 Linking level: The uniqueness of this new media viz., its interactive access of information in terms of blocks of text, images or audio is made possible by means of electronic links. So the information that gets displayed one after another will depend entirely upon the choice of the link relationship. This makes possible for the user to interactively brouse, explore and navigate through large knowledge bases. Since each frame may be connected to a number of other frames, the user can follow a great variety of paths through the given material (Hammond, 1989). This necessitates one to look at interactive multimedia as being able to connect blocks of information in non-linear, multidirectional ways. The linking can also be based on an open system and let the user dynamically generate these links as and when required (Hall, 1994). 3.1 Information linking through its content Essentially, the user makes use of these links to navigate himself through the given knowledge base. The links influence the context they convey and one would expect the links to signify coherent, purposeful and above all, useful relationships(Vaananen,1993). Work is being done in trying to link information semantically through its content rather than as syntactic blocks of information. This requires the links to be able to understand semantically each form of information (Jain, 1994). 3.2 Relating information Interactive multimedia should be seen as a means of usefully relating pieces of information by means of flexible links, so that the essence of the interactive multimedia lies in the way such linking is achieved. The linking of information in the form of text, images and audio can now be expected to give rise to a new way of reading them. Since interactive multimedia emphasises on interconnectedness, it facilitates in a cross-relating of different kinds of data and would simultaneously encourage habits of relational thinking in the user. Hypermedia seems to be closer to the way we think and access information in an associative manner rather than in sequential or linear modes (Vaughan,1993). 3.3 Exploration of information Interactive multimedia's exploratory nature makes it a 'discovery' type of system such that an user can participate in unconstrained exploration. This media should be perceived as a device that turns the information transfer process interesting by stimulating the user to think and explore them. This can also allow for intensive experiencing in the information space(Vaananen,1993). Because of its exploratory nature interactive multimedia shifts emphasis on learning strategies from that of 'learning by memorization' to that of 'learning by reasoning' and 'learning by discovery'. It is possible to aim at a student-oriented, free learning approach, which attempts to let the student have maximum control over the choice of learning strategies and media so that the learner directs the system, instead of a vice versa situation. 3.4 Levels of interaction Interactive multimedia allows for different levels of interaction, with the information at the stage of accessing the programme. It can also be programmed to suit different levels of learning abilities of the users at their own discretion. A child who wants a simple viewpoint to a student seeking in-depth information can access the same programme depending on the user's ability to understand the knowledge base. The same facility can be used to suit the requirements of different kinds of users. For example a programme can have the facility of multilinguality and be accessed in the language one is familiar with. Interactive multimedia should be seen as a programme with an built-in heirarchy such that information can be structured to remain at various levels and information accessed at various levels and for different categories of user. 3.5 Path of information The process of retrieval of information in interactive multimedia is directly related to the way information is connected by means of links. Each block of information in the form of text, images or audio can be connected to a number of other blocks of information and these in turn to a number of other blocks of information and so on. Unlike a book or a film, in interactive multimedia the information retrieval is multidirectional depending upon the numbers and ways the linking is done. This facilitates many different ways of interacting with the given knowledge base. 3.6 Information linking The path of linking the information can have three variations. Linear -information linked to another information Branching - where one information unit is linked to more than one choice o Uniting - where information is linked such that a choice of more than one combines to form another These three basic linking variations can however be combined in a great variety of ways. Some of these are: Tree - information branches and then further branches. An example is a story with many endings Parallel - where information is structured along parallel paths such that one can move from one path to another. Examples are that of a city tour, multilingual access to information, etc. Nodal - section after section leads to well-categorised linking . An example is that of a virtual market, where one goes to a particular section and finds related items. Cyclic - such as where information is connected in loops so that one comes back to the earlier stage. It is not as if multidirectional linking does not have any drawback. Some of these being that the user can get lost or the user might not gain an overview of the given material and miss relevant sections, and might even have difficulty in finding the relevant information. The navigation through information has to be carefully planned and designed. 4.0 Organisation level Many available applications using interactive multimedia are modelled on the linear structure of the book (with options for animating diagrams), and often this appears to be merely an exercise in substituting the form of pages (and pages seem to be best suited for a book). The distinctiveness of this new media offers new possibilities. It is easy to mimic or imitate the old ones before the extent of the potentialities of interactive multimedia are fully realised. 'New media extends our sense of reality and it looks to its predecessor for language and conventions, referring and adopting its characteristics until its unique characteristics can be explored and codified' (Cooper,1989). This means also that interactive multimedia should be understood as a unique, separate medium in order to realise its potentials; this should be done by understanding the inherent codes and conventions and not by treating it as an extension of or a replacement for the other medias. 4.1 Environment for information The interaction with information becomes easier to understand if in some sense they were to reveal their functions visually through their formal structure or configuration. That means the information though connected in complex configurations, has to be perceived as having a simple structure to interact with. To make it easier for the user to navigate in this media it requires that the information is organised in understandable patterns. It is possible to organise different blocks of information and present them in an easily identifiable environment. To make it convenient to remember this virtual environment can be related semantically to something that is of a similar nature and is familiar in the world of the user. Communicating this semantic dimension of the product could provide a means to identify it, personify it and integrate the product into our own world of meanings. 4.2 Metaphoring the information One of the possible devices that a designer could resort to is in the use of a metaphor, whose 'basic premise is the juxtaposition of familiar elements in unfamiliar ways, the connecting of ideas and things not previously connected, to serve as the mechanism in order to constitute new meaning and in turn to gain in expressiveness. The process and result of designing with the metaphor is that elements undergo a change in losing their familiar meaning and contexts and recombine to produce an entirely new knowledge'(Johns, 1984). A successful example is the desk-top metaphor used by the Macintosh range of computers complete with folders and files lying around a virtual workplace and the use of further sub-categories of metaphors like the 'dustbin' for discarding information, or the 'bomb' to show that a program has hung up. 'The use of metaphors seems to be able to aid the process of interaction by providing terms, images and concepts that are esily recognised, understood and remembered' (Marcus, 1993). 4.3 Visualising the information Below are mentioned a few models that could serve to organise information in a virtual environment as a layer - the display surface is treated as a two dimensional surface without any depth. Information can be organised in this as seperate windows. The windows need not necessarily have a regular border and can be in the form of the objects that are displayed. This has similiaity with that of a page from a book. as active layers - information is displayed in many layers one above the other. It is possible to activate any one of the layers. There is no definite border for these layers and it takes the shape of the object that is being displayed. as an extended surface - the physical surface of the the display is extended virtually in planar dimensions. It is as if one is looking at a part of a large scenario. It would be possible to move along the surface and access the information. as enclosed space with virtual depth - it is possible to construct three dimensional spaces inside the display area. The information is accesible along these virtual spaces. This can be in the form of a path. walkway, building etc. as extended space with virtual depth - The information itself can be present in three dimensional virtual space but without any boundaries. One would be able to access this while moving around in this space. 5.0 Variety of Information Interactive multimedia facilitates mixing up of information from a variety of sources - video images, animation illustration, diagrams, still and moving text, spoken words, music and recorded sound. For example, the display of text on the screen can be followed by an animated diagram which dissolves into a realistic moving image, accompanied by audio sounds. The ability of the interactive multimedia to select information from a variety of the medias should be recognised as an important factor in the process of communication. 5.1 Appropriateness of the media The integration of the computer with the medias offers an additional amount of interactiveness for the user. The media now becomes pliable to accommodate upto an extent, the ease of its use and operation and responds individually to each decision, choice or request made by the user. In sum, interactive multimedia should be constructed as a medium that has the capability to choose blocks of information from one medium or a combination of medias such that the selected form of information is the most appropriate to the communication of a given message. These new medias as products of on-going and future technologies, can now create potentially new relationships between these on the one hand, and the user and designer (or author) on the other; and which, in turn, may result in a change or modification of the existing media characteristics. 6.0 Constructing the Interface The design process in the design of an interfaces, needs to be conceived as a physical activity in terms of a development across our sensory capabilities; as a relational process in terms of a linking between and across of information; as an organisation of its various elements in terms of an arrangement across space; and appropriate amalgamation of the various medias.The synthesis of the characteristics of these variables define the syntax of an interface.The diachronic nature of an interface neccesscitates that we look at the aspect of time as a design dimension. Interaction with a product means that we need to organise the sequences of events that make up the interface, and design it from this temporal point of view. This would require that we break up the activities in the form of a story board, something akin to working on a script, and then analyse each of these activities and propose appropriate design solutions; except that to visualise a story with multiple connections in virtual depth would neccessitate developing an ability to visualise information in virtual space. Most activities of design have dealt with the aspect of organising shapes in either two dimensional or three dimesional space. Interface has to deal with the display screen with its virtual depth which can access information randomly from hidden layers. This brings to design a way of organising information in terms of different layers and levels and these have to be perceived in virtual space. . All indications are that designers who have until now been involved with different medias will have to become part of the team that executes the design of interfaces. This means that those with the knowledge of the language of graphic communications, films, animation etc., will now have to morph with aspects of problem-solving of an interactivity that has begun to include the dimension of time. The above-mentioned thoughts are only an indicator of the efforts that we must make in this direction. Acknowledgements The author would like to thank Dr. Ajanta Sen Poovaiah for her suggestions and other assistance with this paper. References Bonsiepe G., Interpretations of human user interface. Visible Language, XXIV 3/4, 1990. Cooper M., The New Graphic Languages in Design Quarterly, pp 4-5, DQ 142, 1989. Hall E. T., The Hidden Dimension, Doubleday and Company Inc., New York, 1966. Hall, W., Ending the Tyranny of the Button, IEEE Multimedia, Spring 1994. Hammond N., Hypermedia and Learning: who guides whom? in H. Maurer (ed.) Computer Assisited Learning, Springer-Verlag, pp 167-180, 1989. Jain, R., Semantics in Multimedia Systems, IEEE Multimedia, Summer 1994. Johns, B., Visual Metaphor: Lost and found, Semiotica, Vol 52 -3/4, 1984. Kolli,R., From notes for a 'Workshop on Interaction Design' at IDC, 1992. Manzini,E., Interactivity, Ohagono, March, 1993. Marcus, A, Human Communications Issues in Advanced Uls, Communications of the ACM, April, 1993. Poovaiah, R. Hypermedia for education: the Rhetorics of a new media, Desg Issues-3, IDC publication, 1993. Schank,R.C., Active Learning through Multimedia, IEEE Multimedia, Spring 1994. Sunaga,T., Education for Interface Design: Designing Change and Message, Industrial Design 157, (Japanese), 1992. Vaananen K., Interfaces to Hypermedia: communicating the structure and interaction possibilities to the users, Computers and Graphics, Vol 17, No. 3, pp 219-228, 1993. Vaughan T., Multimedia- making it work, Osborne McGraw-Hill, 1993. Weiser, M, The Computer for the 21st Century, Scientific American, September, 1991. - The above paper is published in 'Construction of Interface for Interactive Multimedia' in IETE Technical Review, Vol 13, No. 6, Nov-Dec 1996	Contact Details: Industrial design Centre, Indian Institute of Technology, Powai, Mumbai 400076 Telephone: 091-22-5767820 091-22-5767801 Fax: 091-22-5783480 091-22-5767803 email: ravi@idc.iitb.ac.in . Ravi...