ABSTRACT Web-based, Dynamic and Intelligent Simulation Systems

S.VOSINAKIS, T. PANAYIOTOPOYLOS, J. KALLIGATSIS, K. KABASSI

University of Piraeus, Department of Informatics Knowledge Engineering Laboratory

80 Karaoli & Dimitriou str. 18534 Piraeus, Greece (+301) 4142038

E-mail: spyrosv@unipi.gr, themisp@unipi.gr,

jkalliga@unipi.gr, kkabassi@unipi.gr,

On the other hand, Logic Programming (LP) has shown rapid progress in areas like Problem ABSTRACT

solving [7], proof systems [8,9], temporal reasoning and planning [10,11], etc. Recently In this paper we present a method and an

many attempts have been made to put together architecture for constructing intelligent

technologies like Logic Programming, Object - simulation systems over the World Wide Web by

Oriented Programming, as well as Virtual integrating different technologies such as Logic

Reality [20, 22, 23]. Therefore, a need for a step Programming, Object-Oriented Programming

further was emerging gradually and the last few and Virtual Reality. The proposed approach is

years it became rather imperative: The better explained by the presentation of a multi-integration of all these technologies. agent VRML game where agents appear to

behave in a dynamic and intelligent way.

In this paper, we present an architecture, which can be used as a highly expressive tool for

KEYWORDS

combining the advantages of virtual reality and intelligent behavior. The proposed methodology

Virtual Reality, Intelligent Systems, Modeling

can be applied to various kinds of applications.

and Simulation, Web-Based Applications.

As a case study, we present a program that simulates the game of musical chairs.

1. INTRODUCTION

During the last few years VRML has accomplished extraordinary advances towards the field of three-dimensional visual representation [1]. It has motivated people to construct complex worlds where the user can virtually navigate and explore them. Empowered with Java, with the use of the External Authoring Interface [2], VRML has abandoned the static modeling style and is capable of complex object manipulation. Quite recently, attempts have been made to integrate Desktop Virtual Reality with simulation to produce a higher level of interaction and visualization. Researchers have tried to model concepts such as ‘Virtual University’ [3], three – dimensional representations of chemical structures [4], ‘Virtual Laboratory’ [5,6] etc. In spite of all these, VRML can still not handle dynamic behavior of virtual objects.

2. INTELLIGENT VRML OBJECTS

Standard Logic Programming languages produce only text output. E. Denti, A. Natali and A.Omicini [12], use Java as an effective GUI between the user and a logic program. Although this is a worth-mentioning effort, they mostly use Java for its network facilities.

The availability of logic programs on the Internet is the subject of a research done by S.Beltagy, M.Rafea and A.Rafea [13]. They follow an architecture that uses a Java front end for visualizing logic programs. Nevertheless, the visualization is still static in the sense that the output of a logic program is produced in text. An interesting application of the current research towards visualizing logic programs would be the introduction of ‘intelligent behavior’ of virtual objects, i.e. the control of the virtual objects’

-1-

actions using techniques from the field of Artificial Intelligence. Jeffrey Coble and Karan Harbinson, who present a Multi – Agent architecture for Virtual Environments (MAVE) [14], propose such an approach. They describe an effective platform for groupware and coordination of information systems. Although they have managed to visualize complex processes, such as planning, they insist very strongly upon the implementation part, skipping the clarification of the methodology that led them to the specific result.

Other researchers are dealing with the visualization of complex behavior of synthetic characters using methodologies based on Artificial Intelligence and Intelligent Agents theories. They focus on systems that implement real-time virtual humans [15] and their applications, as well as the representation of autonomous intelligent agents as avatars in virtual worlds [16].

We have also done some preliminary work on incorporating artificial intelligence techniques into VRML worlds. In [20], we presented an intelligent agent framework for visualizing intelligent agents. However that approach was rather problem-oriented. More recently, [22], we have tried to develop a Virtual Reality Intelligent Agent Language. The proposed language however, has not been completed, as it must take care of many complex issues concerning VRML objects behavior. An attempt to describe a Distributed Intelligent Virtual Agents architecture has been made in [23].

3. MOTIVATION AND SPECIFICS OF INTELLIGENT AND DYNAMIC BEHAVIOR

The majority of the VRML worlds is either static or includes predefined (predictable) objects’ behavior, lacking realism and failing to capture user’s attention. In an interesting VRML world, objects would be able to adjust their behavior to the alterations of the environment and decide for their actions according to a set of rules. Such a behavior is both dynamic because the sequence of an object’s actions depends on the environment, and intelligent because objects follow a certain strategy for their actions. In order to find a satisfactory solution for this problem, a number of protocols that enable the

-2-

manipulation of VRML worlds by programming languages have been introduced. The most promising one is the External Authoring Interface [2] that allows a Java Applet to control VRML Nodes and alter their attributes.

We call standard VRML animation sequences, primitive animations, because they are considered basic elements of more complex ones. Primitive animations normally concern the transformation of parts of complex objects. Composite animations, on the other hand, concern the synchronized motion of all the parts of complex objects. Composite animations are primitive actions of artificial intelligence planning systems. Such systems tend to use primitive and composite actions (which are defined in terms of primitive actions) and, with the aid of a planner, they generate a sequence of actions in order to achieve some predetermined goals.

Intelligent and dynamic behavior must first be defined in an abstract level, then it must become more specific in a lower level and finally it must be visualized. A planner could handle all decisions using its internal world representation and a set of rules that constitute the reasoning mechanism. Such a system defines composite and primitive actions and generates a sequence of such actions. Each primitive action is received by an intermediate unit, which produces one or more composite animations. These sequences are finally visualized as a synchronized set of primitive motion.

Let us assume that the planner has decided that a character must move to the next room. This could mean that it must walk through the room without bouncing on objects, open a door, and walk towards a chair in the middle of the next room. The intermediate unit receives the decided actions and produces composite animations. The visualization unit analyzes this animation at the lowest level as primitive animation sequences.

4. ABSTRACTION -

SPECIFICATION - VISUALIZATION

The proposed architecture consists of three different parts: the Logic Unit, the Interface Unit and the Visual Representation Unit (fig. 1). The logic unit handles all decisions and actions in an abstract way. There is a solver, i.e. an engine that produces the result of a problem and a

knowledge base (KB) containing the rules and facts that describe the specific problem. The KB contains two kinds of information: the domain knowledge, that describes the world and its objects, and the strategic knowledge, that describes the set of rules the solver has to follow in order to decide for an action.

Respectively, the Visual Representation Unit contains a Virtual Reality Management Unit, a 3D Engine that displays and manipulates the virtual objects that take part in the problem. It can execute a set of instructions that determine the geometric transformations on the virtual objects and its output is based on a 3D-Models database. Static models are not affected by any possible action (e.g. a wall), while Dynamic can change their attributes in real time (e.g. a human).

The communication between the above mentioned units is accomplished through the Interface Unit. It instantiates the objects defined in the Logic Program using its own objects base, receives an abstract notion of the action sequence and transforms it into specific instructions. The Interface Unit consists of three parts: the Communication Unit, which is responsible for the interaction with the Logic Unit, the 3D Interaction Unit that handles the geometric transformations and the Object Instantiation and Action Management Unit (OIAMU), which coordinates the above units as well as transforms the abstract orders produced by the Logic Unit into specific animation.

The main purpose of the Communication Unit is to interact with the logic unit. It receives messages from the Solver and transmits them to the OIAMU and vice versa. This part of the architecture, as well as the 3D Interaction Unit, depends strongly on the implementation of the System and may be subdivided into smaller parts, as described in the next chapter.

The 3D Interaction Unit enables the communication of the Visual Representation Unit with the Interface Unit. It focuses on movement and animation problems, rather than the Visual Representation Unit’s implementation issues.

Finally, the OIAMU carries out the basic translation of the Logic Unit’s orders to a visually perceived presentation. To achieve this, it contains theme specific object paradigms

-3-

(Objects Base). These paradigms analyze an abstract movement order to primitive animations. The animations are kept in a queue structure inside the paradigm, and are activated one at a time. The main OIAMU engine consists of a timer and a chronically sorted queue of Stop Event structures. This list contains Stop Event structures produced by all active objects.

Logic Unit Solver Knowledge Base Strategic Domain KnowledgeKnowledge Interface Unit Communication Unit Object Instantiation & Action Management Objects Unit (OIAMU) Base 3D Intercation Unit Visual Representation Unit Virtual Reality Management Unit 3D Models Base Static Dynamic Models Models Figure 1: The proposed architecture A Stop Event structure is registered upon activation of animation by an object. Among others, it contains the termination time of the animation . When the clock reaches this time, the object is notified. The object’s response depends on whether it has more animation registered in its queue, or not. In the first case, it activates the

next animation. Otherwise, it requests further instructions from the OIAMU engine. The OIAMU engine has also the ability to generate global events by itself.

5. IMPLEMENTATION

Our way of implementing this architecture is based on a client/server motif. The Interface and Visual Representation Units could be on the client side and the Logic Unit on the server side. This approach enables an arbitrary number of remote clients to visualize the simulation over the World Wide Web.

One possible implementation is to use Prolog/Java on the server side and Java/VRML on the client. The Java client part could be an applet so as to be able to connect to the VRML world. The server part could be a Java application and use a library to handle Prolog files, such as Jasper [17]. In this approach, the Communication Unit should be split into Client and Server side Communication Unit, both implemented in Java.

The system’s units cooperate through a sequence of network messages. This scheme provides for the autonomy of each unit, so that it is easy to regenerate the application for a different scenario.

6. A SIMPLE EXAMPLE: MUSICAL CHAIRS

The example presented here is the known game of musical chairs. This game is played in turns. When the music begins n players run around (n-1) chairs. When the music stops, each player races to his/her nearest chair. At least two players will be heading for the same chair. One of them arrives there first, so the other looks for another vacant chair. The player who remains standing is disqualified.

The logic unit corresponds to a compiled prolog program that any player can call in order to decide what to do next. In this implementation, the logic is common for all the players, but their physical abilities (e.g. speed) vary. The interface unit uses two special classes: Chair and Player. These define the behavior of the corresponding objects by providing a set of methods that accomplish complex animations. A very large

-4-

VRML file contains the world’s, the players’ and chairs’ data and the VRML animation structures.

Picture 1: A sample screenshot

The main engine is based on three loops that terminate on occurrence of special events. The first is terminated when there are no more actions, the second is terminated when a generic Stop Event has occurred (e.g. music has stopped), and the third one is terminated when all but one players have sit.

7. CONCLUSIONS AND FUTURE WORK

In this paper we discussed about combining artificial intelligence and 3D visualization. We proposed an architecture for creating dynamic and intelligent behavior in a virtual world and described a possible implementation.

Since our architecture and methodology is proposed for visualizing actions, we could apply them to intelligent Agent architectures [18,19]. A similar work has already been done [20], but the methodology had not been stressed well enough. Visualizing an agent’s actions is a very important issue as it raises matters concerning a more natural human – computer interaction. We could also expand that thought and include communication of intelligent agents and visual representation of it. We are also researching the possibility of extending our architecture to introduce prolog-driven Intelligent Virtual Agents in large virtual environments [22]. We plan to develop a generic tool for the description and creation of such systems in all levels of abstraction.

Interactive stories are also subjects of great interest [21]. Our methodology could apply to the formation of such stories and possibly will have to take in mind the user’s participation. Our current work does not include user feedback, but it could be easily modified to do so.

REFERENCES

[1] ISO/IEC1 4772-1:1997, VRML97

International Standard. The VRML Consortium, http://www.vrml.org

[2] Marrin, C., External Authoring Interface

Reference. Silicon Graphics Inc., http://cosmoplayer.com/developer/eai [3] Panayiotopoulos, T., Zacharis, N.,

Vosinakis, S., Intelligent Guidance in a Virtual University. IMACS International Symposium on Soft Computing in

Engineering Applications, SOFTCOM’98, Athens, June 1998, appears also in

Advances in Intelligent Systems : Concepts, Tools and Applications, (S. Tzafestas ed.), Chapter 10, pp.109-119, Kluwer Academic Publishers, Netherlands, 1999.

[4] Krieger, J., Doing Chemistry in a Virtual

World. Chemical & Engineering News, December 9, 1996, 35-41.

[5] Panayiotopoulos, T., Vosinakis, S.,

Katsirelos, G., Avradinis, N., Interactive experimentation in Virtual Laboratories. CAL99 International Conference, Virtuality in Education, London, UK, March 1999. [6] Dede C., Salzman, M., Loftin, B., Ash, K.,

Using virtual reality technology to convey abstract scientific concepts. In M. Jacobson & R. Kozma (Eds.), Learning the sciences of the 21st century: Research, design and implementation of advanced technological learning environments. Lawrence Erlbaum, Hillsdale, NJ, 1999.

[7] Breuker, J., Problems in Indexing Problem

Solving Methods. Workshop on Problem–Solving Methods for Knowledge– Based systems in Connection with the Fifteenth International Joint Conference on Artificial Intelligence (IJCAI-97) Nagoya, Japan, August 23-25, 1997.

[8] Sofronie-Stokkermans, V., Representation

Theorems and Automated Theorem Proving in Non-Classical Logics. Proceedings of ISMVL-99, 1999.

-5-

[9] Felty, A., Thory, L., Interactive Theorem

Proving with Temporal Logic. Journal of Symbolic Computation, April 1997. [10] Panayiotopoulos, T., Gergatsoulis, M.,

Intelligent Information Processing using TRLi. 6th International Conference andWorkshop on Database and Expert Systems Applications, DEXA'95, London, U.K., September, 4-8, 1995, appears in Proceedings, N.Revell, A. M. Tjoa (Eds.), 1995, pp.494-501.

[11] Marinagi, C.C., Panayiotopoulos, T.,

Vouros, G.A. Spyropoulos, C..D., Advisor : A knowledge-based planning system. International Journal of Expert Systems, Research and Applications, 9(3), 1996, pp.319-355.

[12] Denti, E., Natali, A., Omicini., A., Merging

Logic Programming into Web-based technology : a Coordination-based

approach. Proceedings of 2nd International Workshop on Logic Programming Tools for Internet Applications, Leuven, Belgium, July 1997.

[13] El-Beltagy, S.R., Rafea, M., and Rafea, A.,

Practical Development of Internet Prolog Applications using a Java Front End.

Proceedings of 2nd International Workshop on Logic Programming Tools for Internet Applications, Leuven, Belgium, July 1997. [14] Coble, J., Harbison, K., MAVE : A Multi –

agent Architecture for Virtual Environments. 11th International

Conference on Industrial & Engineering Applications of Artificial Intelligence and Expert Systems.

[15] Badler, N., Real Time Virtual Humans.

Proceedings of the Pacific Graphics 1997: The Fifth Pacific Conference on Computer Graphics and Applications, Los Alamitos, CA, IEEE Computer Society Press,1997. [16] Elliot, C., Brzezinski, J., Autonomous

Agents as Synthetic Characters, AI

Magazine, Summer 1998, AAAI Press, pp. 13-30.

[17] The Jasper Package. Swedish Institute of

Computer Science,

http://www.sics.se/isl/sicstus/jasper/ [18] Wooldridge, M., Muller, J., Tambe, M.,

editors. Intelligent Agents II, Agent

Theories, Architectures and Languages. Volume 1037 of Lecture Notes in Artificial Intelligence, Springer-Verlag, 1996.

[19] Muller, J., The Design of Intelligent

Agents, A Layered Approach. Volume 1177 of Lecture Notes in Artificial Intelligence, Springer-Verlag, 1996.

[20] Panayiotopoulos, T., Katsirelos, G.,

Vosinakis, S., Kousidou, S., An Intelligent Agent Framework in VRML worlds. Third European Robotics, Intelligent Systems & Control Conference, EURISCON’98, Athens, June 1998, also in Advances in Intelligent Systems : Concepts, Tools and Applications, (S. Tzafestas ed.), Chapter 3, pp.33-43, Kluwer Academic Publishers, Netherlands, 1999.

[21] Sgouros N. M., Tsanakas P.,

Papakonstantinou G., A Framework for Plot Control in Interactive Story Systems. Proceedings of the 13th National Conference on Artificial Intelligence

(AAAI-96), Portland OR, USA, AAAI/MIT Press, 1996, pp. 162-167

[22] Panayiotopoulos, T., Anastassakis, G.,

Towards a Virtual Reality Intelligent Agent Language. 7th Hellenic Conference on Informatics, 1999, Ioannina, also in

Advances in Informatics, D. Fotiadis and S.D. Nikolopoulos (Eds), World Scientific, 2000.

[23] Vosinakis, S., Anastassakis, G.,

Panayiotopoulos, T., DIVA: Distributed Intelligent Virtual Agents, Workshop on Intelligent Virtual Agents, Virtual Agents 99, The Centre for Virtual Environments, University of Salford, pp. 131-134, Issn 1467-2154, Salford, U.K., September 13, 1999.

-6-