ISS Simulation

PSA: Find Item

Release 3 (2003.06.26)

This simulation uses the DigitalSpace Oworld library to create a Brahms controlled simulation. By responding to commands sent by Brahms through a HTTP interface, and by providing real-time information to Brahms through the same interface, this system provides a high fidelity coupling between simulation and display systems.

This current build of the simulation and environment does not actually communicate with a Brahms server. Instead, a set of scripts embedded in the web page provide command strings that are fed into the simulation as if they had come from Brahms. However, even with this stop-gap measure, the system is able to provide commands sufficient to perform a search through the ISS using the PSA Agent.

Running the simulation

Once all the model data is loaded for the simulation (indicated by the "Loading..." bar in the lower right corner), the Agents (PSA and Astronaut) will be visible. The PSA will be pointing out the location of the Laptop, while the Astronaut will watch the PSA.

To send the PSA off to find a tool, click the "PSA Search" button on the right hand side of the window. The PSA will then be ordered to move to one of the modules of the ISS, and scan for the required tool. If this is done without detecting the tool, the PSA will be ordered onto another module, and the process repeated until the tool is found. Once found, the PSA will return to the Astronaut, and report the location of the tool.

During this time, the PSA may be affected by fans, pushing it off course, as well as needing to return to charge its battery. It will also avoid the astronauts moving through the ISS, waiting for them to move or finding alternate pathsways to reach its destination.

The Astronauts move about the ISS, the male staying in USLab, the female moving through the modules.

Status Information

Each of the agents will have two status icons floating above them. The topmost one shows the action currently being undertaken by the Agent. The lower one shows what Interactive (tool) or other Agent that is relevant to the current Action. For example, the Astronaut will have a "Look" (picture of an eye) with a picture of the PSA, to indicate it is watching the PSA.

Each of the Interactives will have a status icon, showing the picture of that Interactive. This is to help the user find them.

Additional information can be seen by clicking the "Show Status" button on the control panel at the right of the window. This will show a pane with the list of Settables and Detectables relating to the agents, and their states. Some of the Settables (notably the ones for the PSA) can be changed from this pane, causing the simulation state to change as well. Note that these are queued, and will not take effect until the current action of the Agent is completed.

Moving the tools

The tools are placed randomly through the ISS at the start of the simulation. To move one of these tools, point your mouse cursor at the location you want the tool, and press the Ctrl and Shift buttons. You will see a red dot (like a laser pointer) appear where the mouse is pointing. When this dot is where you want the tool to move to, release Ctrl and Shift, then press the appropriate "Move (tool)" button.

NOTE: Make sure you do this inside the ISS. If you are outside, then the laser pointer will not show up. However, clicking to move the tool will move it, but to a location outside of the ISS, and thus, the PSA may not be able to find it.

Brahms commands

The Brahms commands sent into and out of the environment are displayed in the text box at the top of the window (new commands are added to the top). A > or < sign indicates if the command is being sent from the environment, or from Brahms.

Several of the commands have been extended to provide the capabilities required for this simulation. The most significant is the addition of the "interface" commands. These relate to Settables and Detectables, and are used in both directions. For example, an interface command will set the value of a Settable, or activate a Detectable, while a similar command in the other direction will notify Brahms of a Detectable being triggered.

Below is an example series of commands:

<interface|setSettable|0|projects.issvre.PSA|Watching|projects.issvre.flashlight1
activity|move|118.944999694824|-1|projects.issvre.PSA|Walk||projects.issvre.CentrifugeAccomodation.center|projects.issvre.USLab.center
interface|setSettable|0|projects.issvre.PSA|Watching|projects.issvre.Astro
>interface|detectable|108.944999694824|projects.issvre.PSA|Search|true

In this sequence, the PSA has reported the triggering of its Detectable "Search". This resulted in the setting of the PSA Settable "Watching", making it watch the flashlight when Idle. It was also told that at the time 118... to move from the Centrifuge Accommodation to the US Lab. The -1 informs it to take as long as it needs to (rather then taking a prescribed amount of time, resulting in the PSA moving very quickly).

For more information on Brahms commands used, check the Syntax file.

Simulated Brahms Simulation

This demonstration includes code that simulates a Brahms server. When the environment sends an alert and expects a response, the scripting responsible for sending this data to and from the Brahms server instead produces data of its own, that is fed into the environment as if it had come from a Brahms server.

Below is a psuedo-code description of the processing used in this "fake Brahms", to produce this scenario:

Initial: PSA Idle State = None
PSA Area = USLab
Astro Area = USLab Bar0
Astro2 Area = USLab.Node2.right

Alert received from environment
  Break Alert into sections (type, time, source, property, value)
  if source == PSA
    if property == Idle and value == true
      switch on PSA Idle State
      Move To New Area:
        Select new area
        Set response to "move from PSA Area to new area"
        PSA Area = new area
        PSA Idle State = Search Area
        break
      Search Area:
        Set response to "Do Search Action"
        PSA Idle State = Move to new area
        break
      Search Found:
        Set response to "move close to target"
        PSA Idle State = Return to USLab
        break
      Return to USLab:
        Set response to "move from PSA Area to US Lab"
        PSA Area = new area
        PSA Idle State = Give Report:
        break
      Give Report:
        Set response to "Give report"
        PSA Idle State = None
        break
      Path Failure:
        Set response to "move from Failure Area to PSA Area"
        PSA Idle State = PSA Idle State (Path Failure)
        break
      Power Alert:
        Set response to "move to power point"
        PSA Idle State = Recharge
        break
      Recharge:
        Set response to "Do recharge action"
        PSA Idle State = Return From Recharge
        break
      Return From Recharge:
        PSA Area = PSA Area (Power Alert)
        Set response to "Move from power point to PSA Area"
        PSA Idle State = PSA Idle State (Power Alert)
        break
      None:
        break
      end switch
    else if property == Search and value == true
      PSA Idle State = Search Found
    else if property == PathFailure
      PSA Idle State (Path Failure) = PSA Idle State
      PSA Idle State = PathFailure
    else if property == PowerLevel
      if value == 1 and PSA Power Alert == not triggered
        PSA Area (Power ALert) = PSA Area
        PSA Area = Power Point
        PSA Idle State (Power Alert) = PSA Idle State
        PSA Idle State = Power Alert
        PSA Power Alert = triggered
      else if value == 4
        PSA Power Alert == not triggered
  else if source == Astro
    if property == Idle and value == true
      Select next area, Astro watching target
      Set response to "Dont give PSA LOS alerts"
      Set response to "move from Astro Area to new area"
      Set response to "Give PSA LOS alerts"
      Astro Area = new area
    else if property == LOS PSA and value == true
      Set response to "Watch PSA"
    else if property == LOS PSA and value == false
      Set response to "Watch Astro watching target"
  else if source == Astro2
    if property == Idle and value == true
      Select next area, Astro2 watching target
      Set response to "move from Astro2 Area to new area"
      Astro2 Area = new area
    else if property == LOS PSA and value == true
      Set response to "Watch PSA"
    else if property == LOS PSA and value == false
      Set response to "Watch Astro2 watching target"
Return response

Additional features

GUI Mode

This toggle changes the method used for displaying the status icons for the Agents. In GUI Mode, the icons are always the same size, and always visible.

Mouse Look

This toggle changes how the player/actor is controlled. When active, moving the mouse will cause the direction the player is looking to move. When the mouse is near the edges of the window, the player will continue to turn in that direction.

The player is moved using the arrow keys. The Up arrow will move you in the direction you are looking, the Down arrow will move you backward. The Left arrow will side-step (strafe) left, and the Right arrow in the other direction.

Current Limitations

Moving Tools

Make sure you do this inside the ISS. If you are outside, then the laser pointer will not show up. However, clicking to move the tool will move it, but to a location outside of the ISS, and thus, the PSA may not be able to find it.

Line-Of-Sight Testing

The primary limitation that I have had with this has been the Line-Of-Sight/rayIntersection functions. There are a group of functions designed for detecting points of collision along a line (for example, finding out where a laser beam strikes an object, to place a reflection object/sprite).

A limitation of these functions is related to the way Atmosphere handles loaded models. When doing a rayIntersection test on the world geometry, you are forced to use the ENTIRE geometry, including any Models loaded by the script (be they Atmosphere objects or Viewpoint).
The result of this is you cannot choose to "see through" objects that are dynamically placed, and you cannot tell the difference between a model and a wall.

An example of this is when the PSA attempts to look for a tool. By performing a rayIntersection (line-of-sight) test in the direction of the tool, it can determine if it can see the tool or if there is something in the way. However, you cannot tell what. If the something this test detects is supposed to be transparent (eg the PSA's laser beam), you cannot tell.
Also, the PSA and astronauts have to "look" from a point outside its own body, otherwise it will see its own body, and think something is in the way. This can lead to problems where if the agent is near a wall, they can often see through it (as the offset to avoid seeing themselves places the point they are looking from on the other side).

A place where this is more of a hazard is in path finding. The path-finding routines perform these line-of-sight tests between pre-defined nodes (waypoints/Area) to determine which it is able to travel to, in an expanding pattern until it finds its destination.
However, if something is blocking one of these waypoints (an Astronaut moving temporaily through that space, or even the model of the Agent that is attempting to find the path), these line-of-sight tests will fail, and it will think there is no way to get to the destination point.
In the current implementation this is overcome by continuously retrying if a path isnt found (hoping the astronaut will move). Often this can last a while (especially since the line-of-sight tests are not 100% accurate when working with animated Viewpoints), causing the simulation to appear stalled.

Implementation Issues

The current implementation does not use the Havoc physics engine. This is due to a number of reasons, including inflexibility in the physics modeling (center of gravity is always center of geometry), physics modelling not being aware of changes in a Viewpoints geometry due to animation (at last test), manikin collisions (an astronauts entire body will bounce from the wall, rather then just their arm bending), and speed (particularly during collisions).

Future Directions

Database (Dynamic Code)

Currently this system works using static Javascript. However, it has been designed keeping the idea of using a back-end database in mind. The code is all modular, so that parts could be added and removed "on-the-fly". If for a simulation an agent was only required to do a small amount of actions, only the code for these actions would be pulled from the database/archive, rather then loading every available action.

An example of this would be having multiple astronauts in the ISS. While both astronauts would use the same base code (as both are Agents) and would use some of the same code each (moving, watching other Agents), each astronaut would only be provided the code relevant to what it is required to do. So the astronaut that is never required to "eat can with fork" will not know HOW to "eat can with fork". However, to give that astronaut that ability, the code would simply be dynamically loaded from the database/archive.

Database (Syncronisation)

There is currently no attempt to syncronise different instances of the environment with each other, resulting in each visitor to the environment seeing something different. Using a combination of PHP and a database (as with the Atmospherians Station) it would be possible to provide syncronisation between all instances of the environment.

Compatibility with "official" Brahms Commands

After recently seeing an "offical" Brahms command log, it is obvious that the Brahms Commands that are being used there are not compatible with the Brahms Commands this environment has been designed around. Wether this is due to the age of the log, or that the specification of commands that we have is out of date, is unknown.

Procedural Skeletal Animations

This would allow animations to become dynamic. For example, rather then having a "put down" animation that shows the model placing an object always in the same location, the animation would be dynamicly changed to show the model placing the object where it will actually go. It would also allow animations to allow for collisions. For example, if while drifting through the ISS an astronaut was to brush a wall, rather then either A) their arm going through the wall (non-solid object), or B) their entire body bouncing off the wall (solid object), they would be able to bend their arm out of the way.

Diagrams

Flowchart of current implementation

Flowchart of future implementation

History Of ISS Simulations

First was the PSA moving about the ISS on a random script. 2003/nasa/iss
Second was the PSA moving about the ISS in response to HTTP Brahms Commands. 2003/nasa/brahms
Release 0 added the Astronaut watching the PSA move. 2003/nasa/astro1
Release 1 made the Astronaut an Oworld Agent (and made him move). 2003/nasa/astro
Release 2 tried to fix the syncronous message bug. 2003/nasa/astro2
Release 2.1 fixed the synchronous message bug.
Release 3 is this Release.