DigitalSpace SBIR Phase II Proposal X5.02-9853: Lunar Telerobotics Design Simulation (2006-2007)
See also: PDF version of this proposal and Phase I work on this project (January-July 2004). and new work preformed under this Phase II

Part 1- Table of Contents

PART	DESCRIPTION	PAGE
A	Cover Sheet	1
B	Project Summary	2
1	Table of Contents	3
2	Identification and Significance of the Innovation and Results of the Phase I Proposal 2.1. Innovation 2.2. Results of Phase I Project	4
3	Technical Objectives	18
4	Work Plan 4.1. Technical Approach 4.2. Task Descriptions 4.3. Meeting the Technical Objectives 4.4. Task Labor Categories and Schedules	29
5	Related R/R&D	32
6	Key Personnel and Bibliography of Directly Related Work 6.1 Key Contractor Participants 6.2 Key NASA Participants 6.3 NASA and Non-NASA Advisors 6.4 Biography of Directly Related Work	34
7	Phase III Efforts, Commercialization and Business Planning 7.1. Market Feasibility and Competition 7.2. Strategic Relevance to the Offeror 7.3. Key Management, Technical Personnel and Organizational Structure 7.4. Production and Operations 7.5. Financial Planning 7.6. Intellectual Property	38
8	Company Information and Facilities	42
9	Subcontracts and Consultants	42
10	Potential Applications  10.1 Potential NASA Applications   10.2 Potential Non-NASA Commercial Applications	45
11	Similar Proposals and Awards	48
	Addendum: Capital Commitments Supporting Phase II and Phase III	48
C	Budget Summary	50

Part 2: Identification and Significance of the Innovation and Results of the Phase I Proposal

2.1 Innovation

The design needs of NASA’s Moon-Mars agenda

On January 14, 2004, President George W. Bush enunciated a new vision for space exploration [1,2] which set forth a road map to return humans to the moon by 2020 and prepares for the human exploration of Mars. One year later, the first NASA Robotic & Human Lunar Exploration Roadmap Meeting was held in Houston and NASA’s Chief Scientist, Jim Garvin, and others presented strategies and options for robotic and human exploration of the moon (including 14 day stays at a telerobotically-prepared site) as a test bed for missions to Mars. Figure 1 below shows Dr. Garvin’s Lunar Strategy [3], which was echoed by the other participants including the members of the newly formed Lunar Exploration Analysis Group (LEAG). In each scenario, after the Lunar Reconnaissance Orbiter (LRO) mission of 2008 has identified candidate sites, a series of teleoperated fixed landers and robotic rovers will be deployed to the moon to engage in geologic exploration of the lunar regolith and searching for water ice and other elements for possible In-Situ Resource Utilization (ISRU) in support of human lunar visits.

Figure 1: NASA Chief Scientist Jim Garvin’s Lunar Strategy presented at the first LEAG meeting

As has been enunciated by the lunar exploration community [4] there is a strong consensus that ISRU is necessary for sustainable and affordable human presence in space. The LEAG recently identified the following priorities for early experimental test beds and flight missions [5]:

1. Learn how to move and handle regolith (excavation, drilling, coring, hauling, stacking).
2. Attempt the thermal processing of regolith (e.g., sinter to make pavement).
3. Assess techniques for resource extraction from regolith (e.g., oxygen via reduction of regolith, extraction of ice for fuel and mission consumables).

The LEAG contribution to the NASA Administrator’s 60 day study [6] in July 2005 identified the following value of robotics and telerobotics (abbreviated summarization):

• Mitigates risks to human explorers and increases crew efficiency
• Search for volatiles in permanently shadowed regions, deployment of science packages
• Robotic studies precede human landings at sortie sites and continue after humans leave
• Pre-crew emplacement of equipment and facilities, including Lunar Resource Development plants
• Telerobotics is one component for global access by centrally sited human crews.

With the multiple goals outlined above, there is a perceived need for a minimum of four, and likely to be many more lunar surface missions following LRO and concluding before human crews arrive in the 2016-2020 time frame. If a single-site telerobotically-prepared human base with some ISRU support is opted for, this could easily rise to a dozen or more missions. With a minimum of one robotic lunar surface mission per year starting in 2010, each with substantially differing goals, there is a major design challenge ahead.

A real-time 3D environment that creates plausible simulations of space environments and vehicles can bring an urgently needed rapid prototyping capability to any NASA mission and vehicle design effort. Going well beyond static artist’s conceptions, this prototyping tool will allow distributed teams to iterate designs on a week to week basis crafting operational scenarios with real-time 3D simulations tightly linked to team meetings and teleconferences, web documents, CAD databases and other decision support tools. DigitalSpace proposed and completed the first phase of this SBIR to model the Colorado School of Mines’ lunar bucket wheel excavator (BWE) to demonstrate that a real-time simulation of a lunar surface vehicle is possible (figures 2-3). The BWE was chosen as it is one of the only lunar ISRU-oriented prototypes in existence. The BWE is the lead vehicle of several classes of mining machines which include further design iterations into trenching and boring machines, all likely to be employed in lunar exploration and ISRU.

Figure 2: Dr. Michael Duke with Colorado School of Mines’ Lunar Bucket Wheel Excavator (R. King, T. Muff, M. Duke).	Figure 3: The simulated excavator climbing a radiation-mitigating regolith mound next to a partially buried lunar habitat.

DSS-Prototyper: A collaborative vehicle and mission rapid prototyping platform

During the development of this Phase I work product, DigitalSpace was also a team member in a number of Exploration Systems Mission Directorate (ESMD) funded ECP programs. Through this experience we gained first hand knowledge of how NASA-industry-academia partnerships work and the many challenges of working with geographically distributed team members. In the USC SuperBots project, members were spending weekly teleconference calls trying to verbally describe applications for modular robotics but having difficulty visualizing them and requested DigitalSpace to produce a quick 3D rendering of a SuperBots EVA-support scenario (figure 4). In the case of Raytheon’s CE&R, we created and hosted visualizations of a complete lunar telerobotic mission scenario (figure 5) on the web and team members were able to blog in their comments (figure 9). Also during this period, Mark Shirley, COTR for this phase I SBIR effort, identified that there is a need for a platform to present shared 3D real-time synchronized simulations allowing recording, playback and shared annotation on teleconference members’ desktop or laptop computers.

Figure 4: Rapid prototype prepared for USC’s SuperBots NASA ECP project showing a superbot design at work on a CEV EVA	Figure 5: Lunar precursor telerobotics base preparation visualization in support of Raytheon’s CE&R (January 2005)

This cumulative evidence indicated to us that there exists a clear and compelling need for a general purpose real-time 3D collaborative rapid prototyping tool to support the NASA community as a whole. This tool would need to integrate tightly with the synchronous and asynchronous collaboration platforms already in use and which are primarily Web-based. ESMD has standardized on Windchill for documents, ICE for CAD, and Nexiom for ontology. All NASA centers and contractors have tested special purpose tools which will need to be part of collaborative design (i.e. Viz at ARC, DOUG and Trick/ARES at JSC, SEE at MSFC, NEVA at KSC). It is for this need and within this environment that we are proposing the extension of the existing “Digital Spaces” open source platform (called DSS) with features to create this shared, rapid prototyping tool. We are calling this proposed tool DSS-Prototyper.

DSS-Prototyper: Bridging the gap between artists’ conceptions and serious engineering

Since NASA’s founding, drawings, paintings, static computer-generated images or simple animated movies have traditionally provided a starting storyboard for mission design. Imagery produced by artists such as John Frassanito and Associates, Pat Rawlings and others are frequently utilized to promote a concept or particular mission but those media are too inflexible to support actual ongoing engineering design iterations and qualification of operational approaches. In addition, NASA and its contractor community will continue to rely heavily on voice conference calls backed up by email and document presentations. The communication of complex mission scenarios is challenging at the best of times. Geographically separated members need a new class of tool to build effective mental models of overviews of proposed mission scenarios as well as detailed design options. We believe therefore that a gap exists between the artist’s conceptions, early written mission scenarios and documents and the engineering commit stage of full computer aided design (CAD) and prototype construction (i.e. “bending metal”). It is in this gap where really clear understanding of complex vehicle and mission scenarios is so important and is where a rapidly iterated 3D prototyping environment could make a major contribution.

A unique and revolutionary new enabling technology

Figure 6: Functional view of a multi user 3D supported rapid design prototyping platform

It has only been in the past few years that ubiquitous and powerful 3D chipsets have become standard in virtually all PCs and many laptops. Therefore, real time 3D rendering is becoming ubiquitous, with the end user able to change their viewpoint and elements of a simulation on the fly. These environments are moving into the engineering mainstream with applications in manufacturing, architecture, urban planning and defense [7]. As a large investor in much of the early real-time 3D industrial applications, NASA has a great legacy to draw on for 3D in mission simulation, training and operations [8]. In the new era of exploration, NASA has a chance to again adopt innovative uses of real-time 3D. NASA can also realize cost savings by supporting development of open source, zero license cost tools driving affordable 3D graphics hardware.

Figure 6 expresses our vision for DSS-Prototyper. Distributed team members utilizing telephonic or online conference calls, email and shared document systems will be able to install a local client capable of rendering a complex real-time 3D simulation. This client will take advantage of the 3D chipsets in team members’ computers to render the simulation rather than trying to “bit scrape” and send the frames through a network, which has been tried unsuccessfully in past NASA teleconferences. Another advantage of a local 3D client is that each team member can drive simulations and change their point of view independently of other team members and run the environments offline (between conferences).

During a teleconference, the networked synchronization of the clients will allow team members to drive simulations that are then shared with others in real time (with the normal network latency), record and play back mission scenarios, and change the starting and running parameters (by changing the initial states and locations of agents and virtual vehicles). In addition, team members will be able to post text commentary to an integrated web-based database interface, engage in text chat, instant messaging, document sharing, video and audio streaming. Whenever required, DSS-Prototyper will integrate with ESMD-mandated web environments and databases and provide tight coupling to specialized team tools.

Rapid prototyping in industry

Virtually every field in which complex machines or parts are built on a short timeline utilizes some form of rapid prototyping. In manufacturing, expensive injection or press molds are machined only after iterations through 3D solid object “printers” which produce physical test articles out of low cost materials in time frames as low as a few minutes [10]. This step greatly reduces errors and improves design prior to the costly machining of final parts. This kind of rapid prototyping is a metaphor for the platform proposed here.

The experience of the DigitalSpace team to deliver the DSS-Prototyper

DigitalSpace has six years of experience working with NASA and has committed itself wholly to the design and construction of the Digital Spaces (DSS) platform. DigitalSpace has built real-time 3D simulations for virtually every type of mission: human and robotic on Mars and the moon, Shuttle/ISS, EVA and IVA, and built interfaces with actively used tools within NASA. All of this work is presented on our website at [10]. Work on DSS began in 2000 and a decision was made to move it into open source in January 2005. Since that time developers from several universities and parts of NASA have begun adding capability to DSS’ plug-in architecture. From our years working with NASA we have acquired a functional level of experience in the provisioning of all of the proposed new collaborative features of the platform: 1) Synchronized collaborative simulation, 2) Recording and playback of simulation sequences, 3) Collaborative real-time visualization and annotation, and 4) Standards-based Web-blogging Interfaces. We will detail this experience next.

1) Synchronized collaborative simulation

DigitalSpace has ten years of experience creating applications for and building servers and clients to support synchronized real-time virtual environments, whether they are “avatar” supported spaces for multi-user community [11] or in-depth industrial simulations. In addition we have six years of MySQL/PHP and other web-database programming which will serve as the back end for the shared environments and a mechanism to integrate NASA web-based tools.

2) Recording and playback of simulation sequences

Figure 7: the Brahms Agent Viewer representation of a DigitalSpace application for driving a synchronized lunar CEV/base and astronaut EVA simulation

We recently completed a CORBA interface layer connecting DSS to the “Brahms” discrete agent engine developed at ARC and USRA [12]. This provides a means to embody agents within a 3D simulation and drive the scenario with a time-step and study, replay or program exceptions and decision branches for a human and robotics mission plan (figure 7). This capability, called BrahmsVE [13], will be utilized in this project. An additional feature suggested by COTR Mark Shirley is being built and allows telemetry-driven recording and playback of any simulation sequence. The first version of this was delivered to a project for ESMD in June 2005. These modes of recording and playback will all be used to delivery DSS-Prototyper.

3) Collaborative real-time visualization and annotation

Figure 8: SimStation Online project (May 2004) with annotated ISS model in browser interface

In the spring of 2004, DigitalSpace worked with Mark Shirley and his team at NASA ARC to develop a version of their SimStation integrated virtual vehicle platform [14] that could be used collaboratively via the World Wide Web. Figure 8 shows the interface of the final version of “SimStation Online” (SSO). In SSO a 3D model of a configuration of the ISS is rendered within a pane of the browser window and various camera views allow users to navigate the model. A click on a part of a part issues a SQL query which brings up a record that describes the part within the station structural taxonomy. Users can also annotate part records with station systems documentation, close-out photography or simple text notes. All annotations are presented and available to other users viewing the model at the same time or later. This kind of direct manipulation and association of a CAD model or real-time simulation with databases and documents will be an integral part of DSS-Prototyper.

4) Standards-based Web-blogging Interfaces

Figure 9: Blog used for Raytheon CE&R lunar telrobotics precursor base visualization

In addition to the synchronous manipulation of the 3D models or simulation, asynchronous web-based blogging or other collaborative web-based information exchange will be provided with DSS-Prototyper. Figure 9 above shows such a website blog delivered as part of DigitalSpace’s support of Raytheon’s CE&R project for NASA. As can be seen here, two visitors to the site added commentary to a movie segment of a simulated telerobotic lunar robotics mission. The commentators’ input then informed our modeling team about deficiencies in the simulation which we iterated by the following week. Such asynchronous tools will be delivered with DSS-Prototyper.

In addition to the technical experience detailed above, we will bring to this project a world-class group of advisors and evaluators (see Part 6) who will help us test and improve the platform using actual lunar mission designs. Lastly, important early adopters such as “Project Dust” will be directly using DSS-Prototyper (see Part 3.2 below) during our second year.

2.2 Results of the Phase I Project

The solicitation for the original “Virtual Exploration” Phase I proposal requested the following:

“…innovation in the way that people will interact with both physical and virtual data sets using multi-sensory displays and interfaces (including force-feedback) to support richly endowed situational awareness and telerobotics. “

The work product of this Phase I: Simulation of a prototype lunar bucket wheel excavator

Figure 10: The CSM’s Lunar Bucket Wheel Excavator concept vehicle	Figure 11: Bucket wheel operating in testbed simulant

The Lunar Bucket Wheel Excavator (BWE) concept vehicle (figure 10) was developed at the Colorado School of Mines under the direction of Dr. Michael Duke, designed by Dr. Bob King and built by graduate student Tim Muff (now at Lockheed-Martin Corporation). This vehicle was tested in a lunar regolith simulant sandbox (figure 11) and its ability to excavate simulant was characterized [15]. DigitalSpace selected the lunar BWE as the focus of this Phase I SBIR for the following reasons:

1. A mission-critical need for telerobotic base preparation will be the handling of lunar regolith for In-Situ Resource Utilization (ISRU) for the purposes of a) producing cost effective radiation shielding for a pressurized human habitat through the partial burying of the habitat and b) the processing of lunar regolith for the extraction of valuable resources such as fuels, mission consumables or minerals for part fabrication.
2. The bucket wheel excavator is one of the most common types of terrestrial mining vehicles used for extracting and moving a variety of ores in high volume. The Colorado School of Mines, having determined that this type of vehicle would be a candidate for critical-path lunar exploration and ISRU base deployment, constructed the vehicle.

The CSM’s BWE is one reference design among many possible and represents one seminal data point. The CSM’s lunar simulant sandbox, based on but not using the simulant JSC-1, is only a very rough analogue of lunar regolith. In addition, the lunar environment of a near vacuum, one sixth of Earth’s gravity, and the thermal and radiation environment were not simulated for the CSM’s sandbox testing. Lastly the communications latency present with lunar teleoperations was not simulated in the CSM testing nor were operator interfaces such as haptic joystick controls typically found in terrestrial mining. Despite these limitations, the CSM’s BWE is the only existing example of a concept lunar excavation robot and serves as an excellent starting point for lower technology readiness level research into ISRU and telerobotic operations. The goal of this SBIR Phase I was to build on the strengths of this work to produce a virtual analog of the BWE in a rough simulation of a lunar environment as a test fixture for the evolution of lunar telerobotic systems that may lead to flight hardware. The specific goals of the SBIR project were:

1. Produce a virtual model of the lunar BWE concept vehicle with expert advisement from the vehicle’s designers and place this virtual model within a synthetic lunar analogue environment, constructed as a 3D model, with simulated lunar gravity and rudimentary surface characteristics.
2. Control the virtual vehicle using force feedback joystick which gives a “heavy equipment operator” feel of the vehicle motion and bucket wheel operation with 2.6 second lunar communications latency.

New additions made to the Digital Spaces platform during this Phase I project

Figure 12: Current Digital Spaces architecture showing physics engine and haptic device extensions

To understand what was accomplished in the Phase I project, we will first explain more about the Digital Spaces (DSS) platform. DSS is organized like an operating system having a low-level foundation layer, a middleware simulation layer, a flexible presentation layer and support for various applications. Internally DSS supports three types of construct: core modules, akin to an OS kernel, plug-ins which are extensions to the Core and other plug-ins by an abstracting API, and libraries, which may be databases of objects or groupings of functions. Figure 12 above illustrates the significant additions of modules, plug-ins and libraries during this phase I project:

1. A physics module which through a library called Gangsta, permits a wide range of open source and proprietary physics engines to drive physics in a common scenegraph.
2. A module for a force-feedback joystick (haptic device).
3. Enhanced full screen 3D rendering of the simulation with good performance and overlaid 2D graphical interface objects (Ogre renderer and CEGUI library). The renderer also supports immersive autostereo displays through NVidia quad-buffer hardware.
4. A dedicated BWE dynamics simulator plug-in to match the behavior  of the physical BWE as described by its designer, Dr. Robert King.
5. Additional modules such as a new Camera Control, Entity State Display and Animation sequencers and playback elements were added. In the same period, a plug-in for the Brahms agent system was completed for a separate project. Brahms will be an important element of a Phase II proposal of this work.

The unique achievement of a multi-engine physics implementation

DigitalSpace determined that for any surface vehicle simulation, physics simulation would play a key part. As there are a dozen open source and proprietary physics engines available on the market, each having differing strengths and weaknesses, we determined that it would be unwise to create a dependency on only one solution. We hired a physics specialist, Ed Jones, who proceeded to build what we believe is the industry’s first multi-engine physics module. Ed created and tested an interface (called Gansta) to four of the leading engines (ODE, Novodex, TrueAxis and Newton) which is now integrated into DSS. Figures 13-16 below illustrate our testing of three of the four physics engines via this plug-in methodology. The before and after simulations (tower of objects falling, robot arms lifting boxes) show the slightly different performance of the three engines running in real time in the same scenegraph. The final work product uses the open source ODE physics engine.

Figures 13-16: three physics engines in before-after scenario testing via the generic DSS plugin

Tying a generic game force feedback joystick into a 3D scenegraph with physics

Another significant achievement of this 6 month effort is that we were able to tie the generic DirectX support for low cost game joysticks having a force feedback (haptic) capability into both the DSS 3D scenegraph and the physics engine. Figure 17 below illustrates how these modules interact to create both a means to drive and a sense the contact forces of the virtual vehicle when deploying its bucket wheel.

Figure 17: DSS architecture extensions illustrating feedback between physics engines and force feedback joystick (3D Input Module).	Figure 18: BWE on autostereo display on which both left and right display frames are synchronized with shutter glasses for depth.

Phase I delivery, test and evaluation schedule

The work product was delivered in stages by DM3D Studios (DigitalSpace’s subcontractor on the project) for alpha testing internally at DigitalSpace in March 2005 and the first on-site evaluation at Colorado School of Mines occurred during a site visit in April 2005. There was a public demonstration given in immersive autostereo mode in Melbourne, Australia in May 2005 run for a week as part of an exhibit on virtual environments in the future of manufacturing sponsored by Swinburne University (figure 18). Following the addition of the physics and force feedback and beta testing at DigitalSpace in June, the whole package was made available for test in early July, 2005 to Colorado School of Mines, NASA ARC, and Stanford University. We will report on the user feedback later in this proposal. In conclusion, the work product delivered included:

1. Open source DSS code base additions in Content Management/Versioning System and our open physics solution are now made available to a wider community on SourceForge.
2. Users Guide and FAQ.
3. Joystick (Logitech® Force 3D Wingman model) and drivers when requested.
4. BWE model and content of analog lunar surface base available to as a stand-alone application.
5. Operating system support was implemented under Windows 2000/XP using DirectX 7, 9, or  OpenGL with the option of using NVidia quad-buffer autostereo projector.
6. Secure site to download installation routines, 3D models and documentation, total disk space requirements after installation: 12MB

Phase I user experience

The interface to the Digital Spaces’ lunar bucket wheel simulation permits a number of operating modalities: 1) camera tracking of the virtual vehicle from an independent viewpoint or tied to the vehicle motion (figures 19, 20), 2) operation of the simulation through a joystick or keyboard-based interface (figure 21, 22), 3) force feedback sensing in the joystick interfaces depending on the machine-regolith contact of the wheels/chassis of the virtual vehicle and of the bucket wheel contact with the simulated regolith, 4) teleoperation simulating the 2.6 second latency representing the communications time from Earth to moon and return (with 1.3 second “history”). In addition, the position and orientation of the virtual vehicle is constantly reported.

Figure 19: the BWE scaling a landing exhaust berm to habitat in analogue lunar base	Figure 20: view with camera tracking the BWE and looking down at base over bucket wheel.
Fig 21: Test of BWE at the Colorado School of Mines with force feedback joystick interface.	Figure 22: Logitech® Force 3D Wingman joystick in operation during the simulation.

We utilized a simple 14 day stay lunar base model from another DigitalSpace SBIR project as the backdrop for the BWE testing. We modeled a series of shallow hills and regolith artificially piled up for radiation and rocket motor ejecta mitigation. Thus the virtual BWE was able to drive through a number of traverses over shallow or steep terrain, sintered level landing and trackways and was able to be tested backing up to a static ISRU processing facility model, which has a hopper bay designed for the BWE to offload material (figures 23-28).

Figure 23: BWE in drive mode over simulated regolith.	Figure 24: BWE bucket wheel “biting in” with simulation of dust effects.

We used a simple “puff” of dust to indicate contact of the bucket wheel with the simulated surface. Colorado School of Mines has been awarded a Phase I ECP project to examine dust mitigation strategies for the moon and Mars and has agreed to participate with us in a Phase II SBIR proposal. Part of this participation will allow us to produce particle simulations of likely dust conditions in the lunar environment. These simulations, associated with lunar vehicles, will aid “Project Dust” at the CSM to study mitigation strategies (see Part 3.2 below).

Figure 25:  Detail view of BWE bucket wheel and arm mechanism.	Figure 26: BWE caught on hilltop showing physics action of double rocker-bogie mechanism. Vehicle is able to back out.

Dr. Robert King of the Colorado School of Mines was the original designer of the BWE and participated in guiding the team to implement the particular dynamics of the physical test article. As well as rover dynamics we modeled a simple ISRU processing facility with an input hopper (see figures below). Having a “target” to operate the virtual vehicle helped users to identify problems such as the design of the ISRU unit and the difficulty of backing up to the unit without colliding with the support stands.

Figure 27: BWE backing up successfully to hopper on ISRU processor.	Figure 28: BWE off normal and colliding with support stand for ISRU processor.

Commentary from the Phase I testing of the BWE simulation

The following is a summary of the feedback from the test group at NASA ARC and the Colorado School of Mines. The Stanford test group was unavailable to complete their evaluation.

1. It would be very useful to have a pattern on wheel sidewalls to be able to determine if we are in motion (and a distinct shape on top of wheels per Mike Sims to match how they track MER's progress visually).
2. Collision and other geometry should be visible in a wire frame view mode.
3. We must improve the joystick interface (providing a definable template ideally) to allow the user to navigate their own point of view with the joystick while the rover drives via keyboard or somehow merge these modes. The SimStation modality (derived from Descent) is a good starting point for a better joystick control.
4. The joystick interface and haptic (force feedback) feedback was crude at best, given the use of sub-$100 game device technology. Due to the cost and custom programming interfaces, we opted not to use higher-end haptic devices such as those utilized at Stanford’s Biocomputation center. Testing with higher caliber human-machine interface devices in a subsequent phase of development could involve technology already in use for mining teleoperations [16].

We believe that the Phase I project has proven the feasibility of producing a plausible real-time 3D visual simulation of a lunar surface vehicle and mission scenario in much higher fidelity. We are confident that DigitalSpace’s prior experience in building collaborative and multi-user platforms will enable us to create DSS-Prototyper, a marriage of the 3D rapid prototype simulation and multi-user collaborative tools.

Part 3: Technical Objectives

The high level technical objectives of this project are:

1.      In year one extend the DSS platform to support collaborative synchronous and asynchronous rapid prototyping of virtual vehicles and mission scenarios, not limited to but focused on precursor lunar telerobotic exploration and ISRU missions.

2.      In year two put these extensions to the test in a series of design exercises for important lunar precursor telerobotic missions engaging our advisor and evaluator teams of experts who are listed in Part 6.

3.1 Year One: Extending DSS to create DSS-Prototyper

Through observation of the dynamics of teleconferences and other NASA/contractor group interactions we have derived the following feature set we believe would merge the world of rapid virtual prototyping with work practice norms at NASA:

1.      Synchronized simulations for a given vehicle or mission scenario, design and engineering team members will run DSS-Prototyper in a synchronous or asynchronous collaborative fashion through the Internet as a supplement to normal teleconferences and physical meetings.

2.      Recording and playback of member-driven simulation or operations sequences: all simulation sequences (whether pre-programmed or driven dynamically by a team member) can be recorded made available to project managers, engineers or participants who may have been absent for a given meeting. These recorded simulation sequences will also provide an important historical log of decisions made.

3.      Annotation: any 3D object or simulation sequence would be annotatable by direct user interaction. Meeting notes, decisions and attached media such as images, audio, video and outside references could all be attached to the 3D models or within the interface and stored in the Universal Model Repository (UMR) now in development under another SBIR project at DigitalSpace (figure 12). For synchronous (online) annotation a directly manipulation model (such as SimStation Online as described in Part 2) and for asynchronous (offline) mode, a web site blog can be used (as described in Part 2). Existing NASA web-based tools will also be tightly coupled in.

Platform improvements to DSS  to create the work product

Figure 29 below presents the total constellation of extensions to the current DSS core and additional core modules and libraries (external open source APIs or standard protocols) that will be added to the platform in year one. We will now briefly describe these modules.

Object Exposure: This module provides other systems and languages access to the DSS internal objects and interfaces, as well as allowing DSS components to access external systems as if they were internal to DSS.

User Presence: This module is responsible for representing both the user's presence in the displayed environment, as well as that of other users. It combines the multiple input and output techniques into a conceptual whole user representation.

Figure 29: DSS platform and proposed new development (bold outlines) to enable DSS-Prototyper

Puppet Interface: This module allows remote interaction with other DSS modules and plug-ins. This gives remote servers the ability to control any aspect of a DSS client instance, including the 3D scene graph, GUI and audio. We present the planned Puppet Interface (PI) XML schema:

Incoming

<telemetry>  - This is the surrounding tag of a telemetry stream, and is used to assure the data format. Specific communication protocols may use different identification methods.

<head>  - This is used to contain human-related information about the

telemetry. This includes:

  <title>  - Human readable title of the telemetry.

  <desc>  - Human readable description of the telemetry.

  <author>  - The author/creation tool of the telemetry.

  <timestamp>  - The time the telemetry was recorded, in YYYYMMDDHHmmSS format

  <units>

    <position>  - Units position data is in

    <orientation>  - Units orientation data is in

    <velocity>  - Units velocity data is in

    <rotation>  - Units rotation (change of orientation) is in

<initialization>  - The "perform at startup" telemetry instructions. It contains <chunk> tags

<keyframe time="" >  Contains <chunk> tags to be performed at the time specified.

<now>  - Contains <chunk> tags to be executed as soon as encountered. These are only useful in streamed/generated telemetry. For storage, these should be converted to <keyframe> tags.

<chunk format="">  Telemetry feeds consisting of chunks. Each of these chunks has a specified XML format, and is passed to any Digital Spaces component that is registered to accept that format.

Each major component will be expected to interpret certain formats. Specific examples of these are the 3DVisuals chunk format, and the GUI chunk format. By sending chunks in the specified formats, the remote telemetry provider is able to control these modules.

Outgoing

Outgoing communication uses the same format as incoming. In most cases, the <now> chunk container will be used (as information is generated). Standard chunk formats will be expected, such as a Physics Collision Event chunk format, or a GUI Interaction Event chunk. The outgoing events that are created are controlled from the input stream, by using a Event Generator chunk format. This format allows the telemetry provider to specify events it wishes to receive hunks for.

Data Transfer and the SILC integration: This module acts as a underlying data transportation system for other modules. It allows both real-time communication of events, such as messages and user representation, and retrieval of stored content from multiple sources, such as models and recorded simulations. We plan to adopt the open source, Secure Internet Live Conferencing protocol (SILC) protocol [17] developed under dual GPL and BSD licenses.

The major features of SILC as will be used in the DSS Platform are:

• Secure transmission of data - This is done by using digital keys to encrypt client-to-server, server-to-router, and router-to-router. Also, additional transmission security can be achieved by using keys only know to the sender and receiver, in addition to the network layer keys.
• User identity authentication - This uses the client-to-server public keys with a challenge-response system to ensure the user is the same person they were before, making identity spoofing impossible.
• Distributed serving structure - Using a server cell structure, the network is scalable, and also allowing related clients to be grouped, rather then spread over the entire network.
• Vehicle and Avatar (user viewpoint) Positions / Movement / Animation, Contact list, Public Text Chat, Private Text Chat, Conference Text Chat, Contacts List, Authenticated Contacts

Additions of major architectural components through partnering and open interfaces

Figure 30: major partner components connected to or planned for connection to DSS

DSS has not been developed in a vacuum. It is an open system designed to integrate with the best extant tools and to adapt “agnostically” to data formats preferred in given projects. As figure 30 illustrates, for some project DSS simply acts as a renderer (via the Puppet Interface) for any Integrated Modeling and Simulation (IM&S) engine. These engines will be identified throughout the duration of this project and may be plugged-in by this interface. In the case of more tightly integrated systems, such as Brahms, DSS provides an intelligent execution of agents and extensive event reporting using a custom CORBA interface. During this project we plan to add (or complete if already started) support for High Level Architecture (HLA) which will provide another modality to integrate with other toolsets. The Viz team at NASA ARC has agreed to work together to create an HLA hook within our scope of work supporting the MODAT ECP project directed by Maarten Sierhuis. Viz provides extensive surface modeling capabilities used in MER. With the addition of SILC multi-user synchronization, DSS will securely support a variety of chat, video streaming, audio and instant messaging clients as well as synchronize its own simulations. The UMR and existing repositories of NASA resources will be supported via web queries.

Overarching technical issues

Significant technical challenges are posed by this project. We will review these next. Risk to the project completion will be reduced by having a fall-back position for each high risk item.

Issues arising in distributed simulations and world state synchronization: We will make reference to several Distributed Interactive Simulation (DIS) implementations and may choose to implement one of the standards from the DARPA/US Navy community. We have worked with the DIS group lead by Don Brutzman for many years and have access to resources there. The greatest technical challenge emerges in world state synchronization of physics-based simulation. Physics simulations generate a high number of contacts and other data events that must be transmitted to other running clients to reproduce a faithful simulation across a network (with latency factored in). A key question here is whether to send time-sliced generated telemetry or contact forces. Sending contact forces creates a dependency on the local client’s physics engine working the same as the driver of the simulation (not likely) so the fall-back strategy here is to send raw telemetry. In the more generic set of issues surrounding world synchronization we have identified the following four levels of implementation (we will implement to Level 3 and attempt Level 4):

·         Level 1: File-based synchronization with stored/replayable simulations which would be startable at the same time stamp in all clients

·         Level 2: Shared/driven camera and pointer indicator for live team interaction

·         Level 3: Telemetry-based low latency synchronous simulation

·         Level 4: Contact physics simulation synchronization

Open connections to other toolsets: Essential to the utility of the platform will be its ability to tie in with commonly used NASA team and conference support tools including ESMD databases and web applications (Windchill/ICE) and other tools that come from centers or contractors. Key to this will be the creation of an ActiveX and/or Java Native Interface for the platform. Web delivery of the platform (through AX/IE browser modes will be important and a goal of this project). While this poses a challenge, we believe we can meet the goal of making DSS a component available for integration to other projects.

Rapidly developed ingestor/emitters: Rapidly developing datastream ingestors and emitters including the loading of 3D model content from a variety of sources and formats and parsing and applying associated data (animation, agent behaviors and time sequences) is a significant task. To a great degree, utilizing CORBA, XML, HLA, HTTP and other existing standard interface couplings permit rapid development and lower costs and risk. The provision of such interfaces and rapid adaptation to new data schemas is essential to the adoption of DSS-Prototyper within and outside NASA. We have adopted a plug-in architecture for these components and independent developers at two universities have already used it to rapidly build two new ingestors for projects there. One NASA example of this would be the integration of Trick/ARES telemetry data available from that JSC tool to drive orbital dynamics of a vehicle in the platform. In a preliminary look at this in spring 2005 we interacted with the TRICK team who generated telemetry data and models. We determined that the task of writing a DSS plug-in to interpret the TRICK data was not a lengthy or costly task. The June 2005 Valador/DON project had us write an emitter for simulation data within the space of a couple of days. We will provide full documentation and tutorials for the platform for independent developers to rapidy develop this class of plugins. An open source alpha release of DSS is available on the web at www.digitalspaces.org along with initial documentation, code samples and tutorials as well as a community forum is essential to make this strategy work. The success of Linux, with its kernel team lead by Linus Torvalds, and thousands of developers contributing, is an important proof of concept that a distributed simulation platform can be stable, feature rich, meeting users’ needs.

Higher fidelity simulation of the lunar environment (open access to physics): Phase I Testers requested the adjustment of physical simulation of regolith for more plausible vehicle/regolith contact, adjustability of regolith density and pouring characteristics, a plausible overall model of soil mechanics and dust generation (modeling the ballistic trajectories of particles seen during Apollo). In Phase II a number of options will be explored, include deformable meshes, discrete element simulation or continuous function volumetric simulations. We will also open the physics engine up to the end users to allow them to set parameters and iterate the regolith and robotics simulations. Several of the team members in Part 6 have the skills necessary to assist us in selecting and creating the optimal methodologies.

No regolith handling was modeled in this project. During the course of the work, the project PI, Bruce Damer, attended a granular materials workshop at NASA KSC to determine the level of difficulty of modeling material flow into the bucket wheel mechanism. This modeling was deemed far outside the scope and technical achievability of the project and so remains a challenge for a future phase. A number of potential partners in research and commercial computational fluid dynamics were identified for the next phase should a granular materials simulation be commissioned. A simple lighting model (single bright point source) was employed for the phase I simulation. Lighting in the lunar environment is more sophisticated with back-scatter off regolith (depending on the reflective properties), very dark shadows and even the effect of Earth-shine. In Phase II, attention would have to be paid to the lighting environment.

3.2 Year Two: Live Testing with Robotics Design Variants in Lunar Environments

Year two of this project will focus on presenting and iterating with our group of advisors and evaluators a set of lunar exploration and ISRU vehicle designs as well as mission scenarios within a variety of simulated lunar environments. Just as the user testing on the Phase I bucket wheel excavator showed, we will be able to gauge the effectiveness of the platform as a collaborative tool and a plausible design environment for these missions.

Example of a DSS-Prototyper telerobotic ISRU simulation: Brad Blair, an advisor and evaluator from the Colorado School of Mines’ Institute for Space Resources, lists the key items of interest to any mission modeling as: optimization of vehicle operations and sequencing (plans), lowering of risk to mission hardware or crew, increasing of capability & mission lifetime, and determination out of mission capabilities. Brad then gave us the following mission scenario which features of ISRU telerobotics for the extraction and processing of water ice (figure 31)

Figure 31: sample ISRU telerobotics vehicles and mission scenario to model.

In this example the excavator is extracting ore bearing water ice, offloading material on to a hauler which carries it some distance to be offloaded again at the ISRU processor which uses a power source such as a radio thermal generator (RTG) to produce hydrogen and oxygen which can be made available to the habitat with a feed line or fed into the hauler to run a fuel cell. In return, fuel cell water can be fed back to the ISRU facility and habitat. The hauler and excavator may be powered by fuel cells or RTGs. There are many factors in the above scenario that could be treated in a simulation including:

1. Using the terrain model, physics simulation and vehicle dynamics to optimize for distance traveled orientation and material motion, limiting power consumption and factoring in redundant systems.
2. Testing and mitigating mobility hazards such as being stuck (immobile yet having a healthy vehicle), overturning, or colliding with another piece of equipment.
3. Thermal and radiation and charge hazards and mitigation: maintaining thermal cooling with timing motion of radiators to remain perpendicular to the sun, handling solar radiation effects on sensors, mitigation of radiation effects, tribocharging and arc-discharging damaging vehicles.
4. Modeling granular materials (lunar agglutinates) and dust and their effect on adhesion, progressive mechanical failure, power drawing friction. Monte Carlo simulation methods could be used here as well as designs of sealed, shielded moving surface or magnetic surfaces and brushes for dust removal.
5. Operator interface modalities and situational awareness could be examined with simulation of likely vehicle teleoperations controls, camera views and the use of “shuttle simulator” type game exercises to create gaming scenarios (ie: multiple system faults challenging players to come up with innovative solutions).

In order for the above scenario to be operationally simulated and a number of participants invited to collaborate, 3D models with dynamics would be constructed, a virtual simulant of the regolith to be modeled (with some aspect of the regolith behavior in excavation and hauling/transfer), the thermal and radiation environment modeled, dust production and magnetic adhesion to vehicles simulated, and models of the productivity of motors, the ISRU processor and other systems operational aspects. The DSS platform would be able to model both vehicles and the physics of the surface contact. Many of the project participants (Part 6) bring existing tools able to model vehicle power and thermal, ISRU processes, radiation and dust. Colorado School of Mines is especially committed to providing us tools that we will be integrating through XML and possibly HLA interfaces as per figure 30 above.

Figure 32: the valley of Taurus-Littrow as seen from the Apollo XVII CSM, the target for our model lunar environment	Figure 33: View of an Apollo XVII traverse to “split rock” in Taurus-Littrow, H. Schmitt on the left, Lunar Roving Vehicle on the right.

Modeling the lunar environment: the Valley of Taurus-Littrow and Apollo XVII: Apollo XVII was the final and most extensive manned lunar surface mission [18]. Harrison H “Jack” Schmitt and Gene Cernan logged 22 hours and 4 minutes during 3 EVAs and traversed 33.5km in the Lunar Roving Vehicle (LRV). The traverses took them to varied terrain and geological formations in the valley of Taurus-Littrow (figures 32, 33). We have determined that the Taurus Littrow terrain would make an ideal environment to create our lunar surface model for the test vehicles and mission scenarios. Not only is the environment both varied and well characterized but Apollo XVII astronaut Jack Schmitt has joined this project as an advisor and evaluator and will be guiding us from direct experience. Please find more about Jack’s contribution in Part 6. In addition, dark Polar Regions or other interesting sites may be modeled.

Figure 34: JPL bulldozer designs	Figure 35: CMU’s Nomad and derivatives in the Atacama Desert (1990s-present)	Figure 36: Lunox studies, early 90s (Pat Rawlings, SAIC)	Figure 37: Student lunar construction robotics at ASCE Earth & Space Conference

Other design sources on the lunar ISRU and exploration theme: Since the late 1970s there has been an increasing interest in the design of lunar reference missions. Figures 34-37 above illustrate just a few of the computer modeled studies and physical robotic research prototypes that will provide a tremendous library of approaches for this phase II effort [19]. Perhaps one of the most inspiring sources is the student lunar construction robotics competition at the ASCE Earth & Space Conference. An important outreach and educational spinoff of this project will be to provide these students and their home institutions with a software modeling platform to support their efforts.

Mission profiles served by the tool

NASA’s new exploration vision will require multiple classes of mission to move human explorers beyond Earth’s orbit. Since 1999 DigitalSpace has carried out a range of projects with NASA, its contractors and others which we believe puts us in a strong position to create a key tool to support multiple missions’ collaborative design needs. We will next examine how this tool will serve critical collaborative design needs for the following mission profiles:

·         Surface exploration and ISRU validation missions

·         Crew Exploration Vehicle

·         Telerobotic and human base operations

Surface exploration and ISRU validation missions: The primary focus of this work is surface operations for lunar exploration and early ISRU valiation. However, this toolset will be easily extended to use for simulations and prototyping of Mars surface reference missions and expect this will be part of Phase III applications.

Figure 38: A rapid real-time 3D visualization of a hypothetical lunar CEV.	Figure 39: Simulated CMG changeout for STS-114 (Raytheon and NBL)

Crew Exploration Vehicle (CEV): The CEV and its associated heavy lift cargo capability possibly based on Shuttle derived technology will bring crews and mission hardware to LEO and beyond in the post Shuttle 2010 time frame. DigitalSpace produced 3D visualizations of CEV for Boeing and Raytheon in 2004 (figure 38) and this tool should be well suited to rapidly rendering designs for CEV in ascent, in-orbit, docking and undocking, EVA and entry, descent and landing. A project for Raytheon and the program office of the Neutral Buoyancy Laboratory at JSC (figure 39) allowed us to experience creating a training simulator for crews in preparation for Shuttle/Station missions (in this case STS-114 return to flight).

Figure 40: EVA in 2005 Mobile Agents test, Mars Desert Research Station	Figure 41: Simulation from 2002 BrahmsVE project modeling FMARS Mars habitat	Figure 42: Annotated simulation of lunar CEV crew EVA to traverse base

Telerobotic and human base operations: DigitalSpace’s prior work (figures 40-42) supporting the Brahms team at ARC and USRA for simulations of activities during their FMARS and MDRS analogue Mars base field seasons [20] allowed us to get an early start in modeling human/robotics and the current MODAT ECP project puts it in an excellent position to support complex human-in-the loop human/robotics missions.

Achieving high Technology Readiness Level (TRL)

Figure 43: TRL roadmap	Figure 44: Corresponding levels of development of a lunar telerobotics development cycle (bottom to top)

As is indicated in figures 43 above, this work is being performed at TRL 2-4. Figure 44 shows the corresponding vision for the advancement of the tool and the development of a full telerobotics pathway to flight. Our involvement in three ESMD ECP projects, with the primary lunar focused effort being Project Dust, allows us the opportunity to advance the platform TRL.

“Project Dust”: Contract NNM05AA88C, Mitigation of Dust and ElectroStatic Accumulation for Human and Robotic Systems for Lunar and Martian Missions

Figure 45: CSM lunar sandbox testbed (under construction)	Figure 46: typical sandbox environment at the CSM	Figure 47: Gary Olhoeft describing the Moon/Mars vacuum chamber at the Project Dust kickoff, Apr 2005

Regolith handling and interaction are within the scope of the Dust Project at the Colorado School of Mines and DigitalSpace is already a project partner.  DigitalSpace and the DSS-Prototyper development efforts can be expected to benefit greatly from interaction with the project’s researchers and research data and will be called up on to help the Project to compile the experimental results, parameterize and load them into DSS-Prototyper’s physics models, and depict the behavior of dust based upon the project’s research data. The Project will be using world-class facilities at the School of Mines shown in figures 45-47 above and provide DigitalSpace with the results of physical testing using these facilities.

Aiming beyond Phase II: a Higher TRL enabling use in real-time 3D in mission operations

Figure 48: Opportunity rover’s five week escape from ‘Purgatory Dune’	Figure 49: DigitalSpace’s physics-based real-time model of the MER vehicle

Looking to the future to modes of telepresence: beyond video and stills: At a high TRL, the real-time, collaborative vehicle and surface simulation could be used as a tool for day to day mission operations. A case in point is the Spring 2005 incident where the Opportunity rover experienced a five week gradual dig-out from a sandy area on Meridian Planum known as “Purgatory Dune” (see figure 48) [21]. In 2004 DigitalSpace had produced a low fidelity physics model of the MER vehicle (figure 49). While in no way was this model sufficient to the task of simulating a solution to the real problem, we could look forward to the day when a rover’s pan cam would generate 3D meshes of dunes like Purgatory that would allow a real-time simulation to characterize it as a potential hazard, and our physics models of those dunes and the rover’s drive dynamics would be able to verify the risk. In the future, “open loop” type uplink commands that JPL uses to teleoperated on Mars would be used within a virtual simulation and compared command-for-command with the behavior of the real vehicle. Over time this kind of command-for-command iteration will increase trust in the simulation to the point that it would be used as a daily tool of mission operations.

The long term vision: We conclude the work product part of this proposal with the vision piece we provided for the phase I. We hope that this vision piece conveys the story of where NASA would want to be, perhaps aided by the tool build through this Phase II support:

It is 2019 and late night in Houston but a brilliant day at Mare Smythii on the Moon. Lunar mission operators at NASA-JSC are easing the first piece of heavy equipment down the ramp and the haptic operator receives a satisfying force feedback "bump" as the big excavator encounters its first piece of lunar surface. The real time topography system flickers to life as the stereoscopic cameras on board the excavator produce a 360-degree model of the regolith terrain in all directions, indicating surface and subsurface anomalies including a boulder the size of a house buried by 3 meters of regolith.

At NASA's Ames Research Center in California, donning a lightweight immersive display the excavator operator gets into the driver's seat. Training for years to learn the highly sensitive haptic cab and accommodate the 3 second round trip signal delay the operator is ready to "sense" the lunar surface herself as she drives the excavator forward. She has weeks of careful work ahead, piling lunar soil thickly on top of pressurized hab modules automatically landed at the chosen site for the base. The human crew is only a year away from entering the modules and a lot of extra shielding is needed to keep them safe from high sun spot and coronal mass ejection radiation spikes.

As she shifts her virtual excavator's bucket down and piles up the first load of regolith, terabytes of data drive the particle system model presented in her heads up display. That model tells her the likely volume and makeup of the load based on the stereo camera's real time analysis combined with input from the radar depth profiling system. Satisfied with the virtual excavator's performance in the model, she pulls the lever for the actual lunar excavator and switches to vehicle video to observe it at work on that first load. By load 99 the operator is in as tune with the excavator, the feel of the lunar terrain and gravity and the job site as if she had been standing there at Mare Smythii with a shovel.

PART 4: Work Plan

4.1. Technical Approach

The project will begin with extensive consultation with expert advisors and evaluators and civil servants at NASA centers, primarily ARC (see part 6). The key task for this group is to advise us early on what they consider will be important features in a collaborative design engineering platform. Features will be organized ranking highest ranking to lowest, and prioritized in a development schedule. We will then look at the most important conferencing tools (validated and used within NASA and/or mandated by ESMD) and develop a strategy, priorities and schedule to integrate them. Lastly we will go back to our experts and ask them to recommend the initial three lunar telerobotics vehicles/scenarios to be able to collaboratively prototype and iterate. Knowing the features and tools framework in advance of setting the mission simulation goals will give us realistic insight about what affordances can be offered to the test group in year two.

Project reference website

The Project Reference Website will be a center for ongoing progress and resources surrounding the project, from the specification and design phase to the sample applications advisor/evaluator team testing and resulting iterations and documentation. The site will consist of the following components:

·         Project goals, timeline, team biographies and contact information

·         Documentary results of each phase of the project, from the posting of interviews to architectural and specification documents to code bases, database schemas and 3D models

·         Listserver for project participants with log of message traffic

·         Executable releases of template applications and related tools and libraries

·         Forms for the reporting of feedback, bugs and other information

·         Links to related team resources, other NASA sites and industry sites

4.2. Task Descriptions

The two year plan for the construction, iterative testing, and final delivery of DSS-Prototyper and its proof-of-concept vehicle and mission scenarios is described here, distinct tasks defined by quarter:

Task 1: Scoping of DSS-Prototyper architecture, feature set, mission simulations: DigitalSpace will engage in interviews with its advisor/evaluator teams and within  NASA to gather input as to the optimal architecture extensions (as listed in Part 3) and feature set on which to implement DSS-Prototyper. We will then prioritize the implementation of existing NASA tools and ESMD mandated environments. We will conclude this phase by going back to our team and garnering at least three mission scenarios to model and collaboratively iterate in year two. This activity is planned to take no more than one quarter.

Task 2: DSS-Prototyper first build:  In the second quarter a model DSS-Prototyper build will feature all DSS extensions in test. This will likely include the secure multi user implementation, hooks with web resources and the simulation record/playback mechanisms.

Task 3: DSS-Prototyper second build: The third quarter of the project will focus on the testing of the internally added DSS components and the provisioning of tie-ins with the selected NASA tools and environments. The bridging mechanism to connect with the NASA environments will likely be the addition of support for HLA, the XML-based Puppet Interface to tie into selected NASA tools, and the containerization of DSS within ActiveX and/or the Java Native Interface.

Task 4: DSS-Prototyper first applications: The fourth quarter of the project will produce the first vehicle/mission simulation applications on the DSS-Prototyper framework for internal testing. The containerization of the environment and testing of HLA and the Puppet Interface will be validated for year two. The first tie-ins with NASA tools will be tested.

Task 5: DSS-Prototyper released for beta test: The fifth quarter of the project will feature our first release of the platform to the advisor/evaluator group and NASA testers. An initial set of tutorials and applications will also be released to encourage any non-DigitalSpace group to attempt to build applications. DigitalSpace will assist any group in NASA or a contractor organization to construct an application on the platform based on the template applications.

Task 6: DSS-Prototyper iterative releases: The sixth quarter of the project will concentrate on iterative improvement of the platform and serial releases of new versions of the simulation applications. It is at this point we will add several additional vehicle and mission scenarios that the advisor/evaluator teams might suggest. We feel this will happen naturally as designs “fork”.

Task 7: DSS-Prototyper Gold Master and release packaging: The seventh quarter of the project will prepare for and release the DSS-Prototyper, all documents and applications for gold master, certifying the platform for use in further applications based on our advisor/evaluator feedback, NASA and other groups that may be using DSS. Iterated vehicle/mission simulations will be finalized in this quarter and packaged for delivery as examples of the DSS-Prototyper at work.

Task 8: DSS-Prototyper delivery: The final quarter will involve DigitalSpace and its partners in the project delivering the DSS-Prototyper, including the source code base (under LGPL licensing) along with tutorials and training materials and the vehicle/mission simulations iterated by the advisor/evaluator team and as reviewed by NASA civil servants. All materials not covered under ITAR restrictions will be published on the www.digitalspaces.org site and made available to any NASA or contractor. We will also offer training and support for any outside projects that might be employing the platform by this time. Phase III commercialization will consist of our support of any content or custom development projects that may be underway at commercial entities.

This following section describes the work schedule for the Phase II effort in terms of our projected allocation of person-hours by labor category by task (Table 1) and schedule by task (Table 2). DigitalSpace work is carried out by distributed team members and will be coordinated and assembled at its corporate offices located near Santa Cruz, California. This schedule assumes a 24 month project duration (over eight quarters).

Table 1: projected allocation by labor category by task

TASK	DESCRIPTION	PI	PM	SE	TE	CD	DE	SS
1	Scoping of DSS-Prototyper architecture, feature set, mission simulations	300	100	100	100	25	50	35
2	DSS-Prototyper first build	200	100	150	100	25	50	35
3	DSS-Prototyper second build	200	100	150	150	150	0	35
4	DSS-Prototyper first applications	200	100	200	150	200	0	35
5	DSS-Prototyper released for beta test	200	100	150	100	100	100	35
6	DSS-Prototyper iterative releases	300	100	150	50	100	50	35
7	DSS-Prototyper Gold Master and release packaging	200	100	200	50	100	50	35
8	DSS-Prototyper delivery	250	150	100	150	0	0	30
	Total	1850	850	1200	850	800	300	280

Where:  PI = Principal Investigator, PM = Program Manager, AD = Admin, CD        = Content Developer, SE = Software Engineer, DE = Domain Experts, TE = Test Engineer, SS = Server Support

Total estimated hours: 6130

4.3. Meeting the Technical Objectives

The development of the DSS rapid prototyping capability meets the technical objectives outlined in Part 3 as follows:

1. Platform will permit both distributed development and testing and submission of vehicle designs and mission scenarios.
2. The staged development plan and involvement of domain experts at critical points will ensure that there will be checks of the architecture, especially in the critical area of the viability of the platform as a collaborative tool and the plausibility of the simulations presented to the advisor and evaluator teams.
3. The on-line accessibility of the project will permit a wide collaborating community to evaluate the platform on a 24/7 basis at their convenience and involve colleagues and other organizations as needed.

4.4. Task Labor Categories and Schedules

The work schedule for this Phase I project is described in Table 6 below.  DigitalSpace work is coordinated from its corporate offices located near Santa Cruz, CA (see Part 8). DigitalSpace development and testing teams are at locations in the United States and internationally. This schedule assumes a two year (8 quarter) implementation and application delivery phase.

Table 2: our projected schedule by task.

TASK	Yr 1 Q1	Q2	Q3	Q4	Yr 2 Q5	apps Q6	apps Q7	apps Q8
1. Scoping of DSS-Prototyper architecture, feature set, mission simulations	¦	d	d*	d	dt	*	dt	*
2. DSS-Prototyper first build	¦	d	d*	d	dt	*	dt	*
3. DSS-Prototyper second build		d	d*	d	dt	*	dt	*
4. DSS-Prototyper first applications		¦	d*	dt	t	t*	t	t*
5. DSS-Prototyper released for beta test			d*	d	dt	t*	dt	t*
6. DSS-Prototyper iterative release				d	d	dt*	dt	t*
7. DSS-Prototyper Gold Master and release packaging						dt	dt	d*
8. DSS-Prototyper delivery							t	*

Where:

¦ = Specification and or Design and Documentation

d = Software and Content Development

t = Sample Applications and Testing/Validation

* = Interim or Final Report, Product Packaging and Documentation

Part 5: Related R/R&D

In extensive research of the field of web-based, collaborative 3D simulation environments and model repositories used for training, design and operations we have found no equivalent project being pursued under contract to the Federal Government. We also affirm that our company has not already achieved the stated objectives. The closest project to the proposed DSS-Prototyper and applications that we have been able to locate is that of Dr. Don Brutzman’s work on Distributed Interactive Simulation (DIS) at the MOVES Institute at the Naval Postgraduate School in Monterey, California. We have agreed to cooperate with MOVES to bring their Extensible Modeling and Simulation Framework (XMSF) XML-based standard (XMSF) into the platform and they will in turn open up new applications within the Navy and other US Government customers.

Biography of directly related work

[1] Office of Exploration Systems on the web at: http://exploration.nasa.gov and the President’s announcement of the new vision for space exploration on Jan 14, 2004 is at: http://www.nasa.gov/missions/solarsystem/bush_vision.html

[2] President's Commission on Moon, Mars and beyond, report on the web at: http://govinfo.library.unt.edu/moontomars/

[3] Garvin, Jim, “Strawman Robotic Architectures, Lunar Architectures” presented Jan 25, 2005 at the NASA Robotic & Human Lunar Exploration Roadmap Meeting 1, January 24-25, 2005, Space Center Houston. On the web at: http://www.moontoday.net/news/viewsr.html?pid=15540

[4] Space Resources Roundtable, LEAG and Lunar and Planetary Institute resources on the web at: http://www.mines.edu/research/srr/ and http://www.lpi.usra.edu/meetings/leag2005/ and http://www.lpi.usra.edu/lunar_resources/

[5] G. Jeffrey Taylor, University of Hawai`I, Lunar Exploration Analysis Group (LEAG), presentation at the NASA Robotic & Human Lunar Exploration Roadmap Meeting 1, January 24-25, 2005, Space Center Houston. On the web at: http://www.hq.nasa.gov/office/apio/pdf/moon/05_leag_debrief_taylor.pdf

[6] LEAG contributions to NASA’s 60 Day Study from communication with Jeff Taylor, Jul 12, 2005.

[7] Multigen Paradigm library of projects on the web at: http://www.multigen.com/solutions/success.shtml

[8] R.B. Loftin and P.J. Kenney, Training the Hubble Space Telescope Flight Team, IEEE Computer Graphics and Applications, vol. 15, no. 5, pp. 31-37, Sep, 1995.

[9] Rapid Prototyping resources from the University of Utah on the web at: http://www.cc.utah.edu/~asn8200/rapid.html

[10] DigitalSpace projects and publications on the web at: http://www.digitalspace.com and publications at http://www.digitalspace.com/papers

[11] B.F. Damer, Avatars: Exploring and Building Virtual Worlds in the Internet, Peachpit, Berkeley 1998.

[12] M. Sierhuis, Modeling and Simulating Work Practice; Brahms: A multiagent modeling and simulation language for work system analysis and design, Ph.D. thesis, Social Science and Informatics (SWI), University of Amsterdam, SIKS Dissertation Series No. 2001-10, Amsterdam, The Netherlands, ISBN 90-6464-849-2.

[13] B. F. Damer, et al, Final Report, SBIR I: BrahmsVE: Platform for Design and Test of Large Scale Multi-Agent Human-Centric Mission Concepts, DigitalSpace Documents, July 2004. On the web at: http://www.digitalspace.com/reports/

[14] Shirley, M., Cochrane, T., SimStation: A Knowledge-Integrating Virtual Vehicle, Virtual Iron Bird Workshop, NASA Ames Research Center, March 31, 2004. Available on the web at: http://ic.arc.nasa.gov/vib/day1/papers/Shirley_Cochrane.pdf

[15] Muff, T., Johnson, L., King, R., and Duke, M.B., (2004), A Prototype Bucket Wheel Excavator for the Moon, Mars and Phobos, Proceedings of the 2004 Space Technology and Applications International Forum (STAIF-2004), Albuquerque, New Mexico, February 8-11, 2004.

[16] Automated Mining resources and publications on the web at: http://www.automatedmining.ca/news.html

[17] Secure Internet Live Conferencing resources on the web at: http://www.silcnet.org

[18] Apollo XVII Lunar Surface Journal: http://www.hq.nasa.gov/alsj/a17/a17.html and [] Harrison H. Schmitt, A Field Trip to the Moon, NEEP602 Course Notes (Fall 1996) Resources from Space, UW-Madison Fusion Technology Institute, on the web at: http://fti.neep.wisc.edu/neep602/LEC1/trip.html

[19] David Kortenkamp, Scott Bell and Luis Rodriguez, Simulating Lunar Habitats and Activities to Derive System Requirements, in Proceedings 1st AIAA Space Exploration Conference, 2005.

[20] Mobile Agents and Brahms publications on the web at: http://homepage.mac.com/wjclancey/~WJClancey/WJCCogSci.html

[21] NASA JPL: Looking Back at ‘Purgatory Dune’ on the web at: www.jpl.nasa.gov/missions/mer/images.cfm?id=1699

[22] Victor Mallet, Miners are drilling to unimagined depths in the search for gold, Financial Times Weekend June 5 / June 6 1999

[23] X. Huang and L. E. Bernold. Experimental Work on Robotic Excavation and Obstacle Recognition, Proceedings, Fifth Topical Meeting on Robotics, Knoxville, TN, pp. 83-88, April 25-30, 1993.

[24] Sony’s Aibo and iRobot’s Roomba  web at: www.sony.net/Products/aibo and www.irobot.com

[25] S. Trentanelli, Armed Robots Provide Glimpse of Future Warfare, RDECOM Magazine, Sept 2003, on the web at: www.rdecom.army.mil/rdemagazine/200309/itf_armed_robots.html

[26] The Contact Consortium, projects and publications on the web at: www.ccon.org

Part 6 Key Personnel and Bibliography of Directly Related Work

6.1 Key Contractor Participants

The following brief resumes introduce management/technical staff members for this phase II project.  DigitalSpace certifies that Bruce Damer, the Principal Investigator, has his primary employment at DigitalSpace at the time of award and during the conduct of the project.

Name:                           Bruce Damer (PI)

Years of Experience:     24

Position:                        CEO

Education:                     Bachelor of Science in Computer Science (University of Victoria, Canada, 1984); MSEE (University of Southern California, 1986)

Assignment:                  Mr. Damer will be the Principal Investigator for the SBIR Phase I effort.  He will coordinate all interaction between DigitalSpace and its collaborators and NASA and other participants, be responsible for all staffing, technical design, and review of work product.

Experience:                   Mr. Damer is the world's recognized expert on avatars and shared online graphical virtual spaces having created much of the early literature, conferences and awareness of the medium. Mr. Damer is a visiting scholar at the University of Washington HIT Lab and a member of the staff at the San Francisco State Multimedia Studies Program.

Name:                           Galen Brandt (Program Management, Marketing)

Position:                        New business development, DigitalSpace

Experience:                   27 years including project management and marketing for Dun and Bradstreet, SUNY Fashion Institute of Technology, DoToLearn.

Assignment:                  Program management including in reviewing all aspects of the work product and documentation. She also has responsibilities in the Phase III marketing of the work product.

Name:                           Dave Rassmussen (SE)

Position:                       Principal of DM3D Studios, subcontractor to this project

Experience:                  10 years experience in virtual world design, skills: 3DS Max, Java, Active

Worlds, Adobe Atmosphere, PHP/MySQL database development

Assignment:                 Dave has served as PM for this phase, directing the team performing 3D modeling and animation, testing and delivery of the project

Name:                           Merryn Nielsen (Lead CD)

Position:                       Partner, DM3D Studios

Experience:                  10 years experience in virtual world design and web page creation.

Assignment:                  Web & graphic design on project, 3D worlds, administrative support.

Name:                           Peter Newman (TE)

Position:                       DM3DS 5 years experience in C++, JS, PHP, HTML

Assignment:                  Chief technical architect of Digital Spaces (DSS) core and extensions.

Name:                           Ryan Norkus (CD & TG)

Position:                       DM3DS 5 years experience Graphic artist, 3d modeler and animator

Assignment:                  Focusing on 3D model construction & animated sequences

Name:                           Bruce Campbell Norkus (TE)

Position:                        15 years experience in C/C++ and Java programming, 3D model formats,
simulation, University of Washington HIT Lab, Dept Oceanography.

Assignment:                  Assisting with DSS platform infrastructure around distributed simulation.

Name:                           Peter Meigs, Alex Grigny de Castro, Dean Strik (TE)

Position:                       Server support and infrastructure, DigitalSpace

Assignment:                  Focusing on the maintenance of DigitalSpace’s server infrastructure

6.2 Key NASA Participants

Key NASA participants in this project have included: project COTR, a number of NASA personnel at Ames Research Center which will continue to be the key NASA center involved in this Phase II work with support by the non-NASA participants.

6.3 NASA and Non-NASA Advisors and Evaluators

To facilitate both the live testing of the DSS mult-iuser rapid prototyping platform and iterating designs for lunar telerobotic vehicles and mission scenarios, DigitalSpace has recruited a team of individuals representing some of the world’s top expertise in robotics, the lunar environment and planetary surface exploration. Each team member has one or more decades of experience working in NASA programs, university research or industry. Many have developed robotic systems employing remotely distributed teams. Several have been involved in lunar and planetary missions.

Advisors will be committed for fewer hours and serve in an advisory capacity for the toolset, the plausibility of robotic and environment simulations and mission designs. Evaluators will be actively engaged in using the platform to iterate designs and pull in resources from their home institutions and collegial networks. Who participates in the advisor and the evaluator groups will be determined by time available in the late 2006 time frame. All of the individuals listed below have submitted letters of commitment or are covered under separate agreements with their institutions. Most of these letters are embedded in table 5 below.

In Year 2 we will deliver our platform for testing to both groups and validate the effectiveness of the platform through the iteration of several seed vehicle/mission designs and lunar environments. We will host a series of conference calls with team members running synchronized simulations. We will engage in individual asynchronous dialogue with team members who will enter comments using the project blog. Lastly, we will engage in a number of site visits, to the major host institutions including NASA ARC, JPL, JSC, Colorado School of Mines and the University of Southern California.

Table 3: Expert advisor/evaluator team participants in the project

Harrison “Jack” Schmitt	Michael Sims	Geoff Briggs	Brad Blair
Masami Nakagawa	Gary “Rod” Rodriguez		Peter Will	Robert King
Jeff Taylor	Ron Creel		Brahms/MODAT/Viz team

The advisor and evaluator group members represent a cross section of knowledge and experience in surface missions, innovative robotics, and the environments that will be encountered in ISRU and exploration robotics on the moon and mars.

Mission-experienced team members

Dr. Harrison “Jack” Schmitt flew on Apollo XVII as a civilian geologist and with Commander Gene Cernan, used the Lunar Roving Vehicle (LRV) to perform extensive traverses of much of the Taurus-Littrow valley in December of 1972. The two astronauts collected the largest regolith sample of the Apollo program and over three days geologically characterized the valley and gained extensive experience operating the LRV and using a host of tools in the lunar environment [18]. Jack will evaluate our platform, the hypothetical vehicle and mission designs and guide us through the re-construction of the Taurus-Littrow valley which we will be building as a backdrop for the project. Ron Creel, now a senior space systems engineer at SAIC, was in charge of LRV thermal system modeling and mission support efforts for the Apollo missions and has recently completed an extremely high fidelity computer graphics reconstruction of the LRV. Ron will bring to bear his decades of experience of rovers and other systems in the space environment. Dr. Michael Sims of NASA ARC is a surface robot mission designer and member of NASA JPL’s MER science team. Mike will bring the experience of MER Mars surface operations to this project as well as years of thought about how to build and fly successful lunar regolith handling robotics systems. Dr. Geoff Briggs of NASA ARC brings his long experience as an investigator on Mariner 9, Viking and Voyager and directing NASA’s Solar System Exploration Program. Geoff is currently involved in reference missions to Mars and is PI for Arctic field tests of a Mars drill with JSC and UC Berkeley. Mike and Geoff will provide reviews of our work representing the NASA perspectives and will not be paid as consultants.

Dr. Briggs was an investigator on the Mariner 9, Viking and Voyager missions while working at JPL from 1971 to 1977.  He directed NASA's Solar System Exploration Program from 1983 to 1990 and led the strategic re-planning of the solar system exploration program following the major mission cancellations in 1981 and the Challenger accident of 1986.  He has led Ames' Center for Mars Exploration since 1992 including the Ames' contributions to the Human Mars Exploration Design Reference Mission.  The focus of his activities is on technology developments in support of the future robotic and human exploration of Mars.  He is PI for the development and Arctic field test of a Mars drill being carried out as a collaboration between NASA JSC, NASA Ames and UC Berkeley.

Lunar architecture team members

Dr. Jeff Taylor is a planetary science faculty member at the University of Hawaii specializing in planetary volcanology, igneous processes and extraterrestrial materials. Jeff is also the chair of the LEAG (Lunar Exploration Analysis Group) which will be carrying out extensive community-based studies in support of NASA’s new Moon/Mars exploration vision and report these directly to ESMD via the NASA Chief Scientist. Jeff is convening several conferences and meetings in the next couple of years. Jeff’s participation in the project will expose our platform and toolset to the widest possible community of evaluators and draw in more expert interest in the methodology of rapidly prototyping and hopefully enable the platform to become a tool of the LEAG community. Brad Blair, of the Institute for Space Resources (ISR, formerly CCACS) at the Colorado School of Mines, is an internationally acknowledged expert in all aspects of lunar resources including the physical challenges of regolith handling and processing, the economics of ISRU and how it impacts whole mission architect built one of IBM's first industrial robots in the early 1980s ures. Brad will be bringing the depth of his knowledge of the space resources topic and community to bear on the project and will help us to get DSS-Prototyper into the hands of a wider user community. Brad will also be assisting in directions for Phase III commercialization in the mining industry.

Innovative robotics team members

Every member of the extended team brings knowledge and experience to the design of innovative robots but there are a some who have a primary focus in this area. Dr. Robert King is Full Professor, Division of Engineering, Colorado School of Mines, and the lead designer of the lunar bucket wheel excavator working with Mike Duke and graduate student Tim Muff. Bob’s focus is all matters involving the dynamics and operability of machine/earth contact. Coming in with a focus on mining engineering, Bob applied his background to the BWE’s anticipated mission in a lunar setting. Bob will be involved in testing DSS-Prototyper and critically evaluating all lunar robotics design with reference to interaction with the regolith. Dr. Peter Will is the Director of the Distributed Scalable Systems Division at USC/Information Sciences Institute and is a Research Professor in Industrial and Systems Engineering at USC. Peter will bring to this project his experience in robotics, 3-D robot vision and solid geometric modeling, tactile and force sensing and an interest in the challenges of lunar robotics and regolith handling.

In-Situ Resource Utilization and tools/methods for surviving in the lunar environment

Gary “Rod” Rodriguez, President of SysRAND, member of the ISR and Program Manager of “Project Dust” at the Colorado School of Mines brings years of experience of system design and ISRU background to the project. Dr. Masami Nakagawa, a professor at the Colorado School of Mines, brings his world-renowned lab and experience in dust and granular materials to the project. Working with Rod as PI on Project Dust, a NASA funded ECP efforts, they will be characterizing the lunar environment and tool testing in a vaccum chamber at the School of Mines.

Part 8: Company Information and Facilities

DigitalSpace Corporation was incorporated in the state of California on August 24, 1995.  DigitalSpace is a company organized to innovate and commercialize in the multi-user virtual worlds and virtual communities market. The company’s business is based on the following concepts:

•      That the Internet and especially 3D virtual worlds, voice and text environments, can be effective meeting places and enable communication, learning and team based projects. DigitalSpace uses these spaces in a proof of concept that a company can base its entire operations on them;

•      as the need for tele-working, distance learning, virtual communities of interest and Visualization grows, so will grow the capabilities of ordinary consumer personal computers to deliver real-time 3D multi-user experiences. It is the convergence between these needs and the capabilities of consumer computing hardware that will create a large industry producing and hosting virtual worlds and communities in the future.

On these two premises, DigitalSpace has provided solutions for dozens of clients to produce both demonstration and fully functional virtual spaces since 1995. See our web site at http://www.digitalspace.com for a portfolio of projects and clients. We will feature a number of them here for their relevance to this SBIR proposal:

Facilities: DigitalSpace’s headquarters is near Santa Cruz California and it currently rents office space in a 2 story building at 221 Ancient Oaks Way, Boulder Creek, California 95006. Additional DigitalSpace US team members have satellite offices in Phoenix Arizona and Rhode Island, USA. DM3D Studios, primary subcontractor to this project, has an office in Maffra, Victoria, Australia.

Equipment used: All DigitalSpace team members and subcontractors have at least one personal computer connected on the Internet (most have IBM PCs, Pentium 3-4 class) with all needed software for 3D modeling, website design and programming.

Part 9: Subcontracts and Consultants

Consultants and contractors:  DigitalSpace will employ a number of consultants and contractors in this project. Facsimiles of their signed letters of commitment and availability are included in table 7 below (DM3D studios personnel are described in Part 6 key contractor personnel above as are our Domain Experts):

Table 5: Subcontractor and Domain Expert Advisor consultant commitment letters

DM3D Studios contracting commitment	Lisa Faithorn, MODAT project, USRA, NASA ARC
Apollo XVII Astronaut Harrison H. Schmitt	SysRAND, covering Colorado School of Mines/ISR
Jeff Taylor, University of Hawaii, LEAG	Ronald CREEL, SAIC
Robert King, Colorado School of Mines	Peter Will, USC/ISI

Part 10: Potential Applications

10.1 Potential NASA Applications

Collaborative rapid prototyping of mission design and higher TRL mission operations: As identified in Part 3 of this proposal, DSS-Prototyper will be a general purpose platform capable of supporting a wide range of missions including CEV to LEO, ISS, CEV to L1, low lunar orbit or surface, and on to Mars. As the synchronous connection to Brahms is being completed in another project, DSS-Prototyper will also be able to support collaborative human-in-the-loop mission simulation such as lunar and mars surface base with astronauts and robotics together. This platform will be usable by the Mobile Agents and MODAT teams at NASA ARC. We anticipate that the platform’s no license cost, open source and open standards basis will make it attractive to the civil service research and development community. By granting full rights to the platform to NASA, DigitalSpace hopes it will achieve the fullest measure of adoption. When a platform is fully propagated, then the main business that it supports is content and services, rather than development and this is our goal.

A key phase III goal is that DigitalSpace become a kernel team and assist and coordinate the continued development of the platform while satisfying a number of projects including ECP programs in which it is a participant: MODAT from NASA ARC and USRA, Project Dust at the Colorado School of Mines, and Superbots at USC and NASA JPL. We hope to interest the team designing Mars Science Laboratory (MSL) in the use of DSS-Proto as well as any mission team that emerges from the new exploration vision. If through this project, DSS becomes a client of ESMD’s ICE repository to store and retrieve the elements of a collaborative simulation platform, then it will add value to both platforms and may find a wider audience. Support for ICE will depend on a number of factors, such as gaining support for constructing secure queries.

NASA educational outreach and Space Camp: Immersive virtual worlds and 3D simulations are a significant vehicle for NASA's effort to communicate knowledge/understanding to K-12 and college/university students on topics surrounding the new exploration vision. In 2004, we proposed DSS for use at NASA Space camp to Ed Buckbee, founder of Space Camp and a version of DSS will be made available, with applications for Space Camp. The platform will also be offered to the organizers of the annual student lunar construction robotics competition at the annual ASCE Earth & Space Conference.

In addition, the Space Exploration Online massively multiplayer online game concept that was proposed by DigitalSpace and Boeing to NASA headquarters in October 2004 could be based on DSS-Prototyper if that project achieves support.

10.2 Potential Non-NASA Commercial Applications

Figure 51: Automated excavator at INCO’s Stobie mine in Canada.	Figure 52: Teleoperation of a drill at the Stobie mine using video and joysticks.

Automated mining: Automated mining as illustrated by the Canadian company MD Robotics automation of the Stobie mine, run by INCO (figures 51, 52) permits an operator to run drills, and excavator/haulers from the safety of a surface office, away from health hazards such as rock collapses and environmental issues like dust, blast ammonia [16]. In South Africa, home of the deepest mines in the world, there is an initiative to develop Ultra-deep mining [22] which is aimed at the 5km (3.5 mile) depths for gold deposits conservatively estimated as equal to the size of those mined in the past 150 years in the region (with a market value of some $500 billion). Ultra-deep mining is restricted by the need to cool miners and equipment from temperatures in excess of 70C (160F). There is also an issue of air pressure and “rockbursts” which is a major cause of fatalities of miners.

Deep mining efforts and automated mining are often referred to as being “space age” which is a true characterization. The INCO mine’s telerobotic operation shown in figure 53 relies solely on video cameras, which do not give the operator a complete understanding of the surrounding rock face. Indeed one of the key issues of automated mining is that the operator needs to have complete situational awareness of where the robot is and how it can move [23]. Brad Blair of the Colorado School of Mines, who will be advising this project, describes the proposed use of stereoscopic cameras to generate a total 3D scene of the rock face, tunnel and situation of the mining vehicle. He states that this would be of tremendous help in the development of automated mining and be useful for training operators and testing robotics designs. We have a high degree of confidence that investment in this Phase II project will yield commercial interest from the mining and construction industry who are investing more and more into automation.

During this Phase II effort we will be in contact, through our advisor team, with research managers at Caterpillar, Inc., Bechtel, and others. The ISR at the School of Mines, will maintain a running demonstration setup for the operational simulations produced by this project and these will be available to visiting researchers and managers from the mining industry. Investment in space-based telerobotics is likely to have a direct payback on Earth in billions of dollars of commercial value and recovered ores that would otherwise be out of reach.

Figure 53: US Army tactical robotics	Figure 54: Hazardous materials security robotics

Defense and security, training and operations applications for teleoperated vehicles: From the DARPA Grand Challenge, to teleoperated robotics in munitions and dangerous missions during wartime (figure 53) [24], to first responder hazardous materials sensing and handling devices (figure 54), and security/patrol robotics, the 21^st Century will surely be the “century of telerobotics”. While goals of total autonomous operations remain challenging (as shown in the recent DARPA competition) and more than a small amount of autonomy granted to NASA’s Mars/Moon surface robotics is unlikely to outweigh the risks, telerobotics combines all the best aspects of the intelligence and experience of human operators with the ability of robotics to go where it is costly or hazardous to send humans.

Figure 55: Sony’s Aibo	Figure 56: iRobot’s Roomba

Learning, toy, and domestic robotics design, online and competition “robot wars”: As figures 55 and 56 illustrate, the popularity and feasibility of consumer robotics is growing [25]. Therefore there is a market for both researchers and product developers, to collaboratively design autonomous or teleoperated consumer robotics for toys, domestic duties and health care applications. Robot “wars” are one of the most popular forms of entertainment in the popular media and robot game competition are some of the finest learning events for K-12 and college engineering students and faculty. Massive multi-player online games are experiencing a large amount of investment and commercial interest. DSS in its multi-user mode will be competent platform for the sourcing of a successful reality-based multiplayer online robot game both as a learning tool and as a pay-per-play tournament environment. We engaged in a proposal of such a game in 2004 and hope to pursue this during the Phase II development of DSS-Prototyper.

Industrial design, training and operations applications: From factory floor automation to security systems, construction, complex environments where humans work in tandem with mobile agents or other autonomous machine systems all need a comprehensive model-based environment with high fidelity 3D re-creation during both design, training and operations phases [9]. Industrial training is a multi-billion dollar per year industry and the DSS platform is uniquely suited to enter this market, running on industry standard platforms.

Part 11: Similar Proposals and Awards

DigitalSpace has been awarded other SBIR projects and is a participant in several other NASA and contractor projects which have allowed us to build other aspects of the DSS platform that set the stage for DSS-Prototyper. Table 6 below provides a review of these projects.

Table 6: Recent similar projects: active, complete and pending

Proposal/Contract Number	Title	Description
NNA04AA17C SBIR 2002-II (active)	BrahmsVE: Proof of Concept for Human/Agent Intelligent Augmentation  (DigitalSpace)	Creation of real-time connection between Brahms and DSS providing visual interface for Brahms agent scenarios
NNA05AA06C SBIR 2003-II (active)	BrahmVE Platform for Design and Test of Large Scale Multi-Agent Human-Centric Mission Concepts (DigitalSpace)	Creation of universal model repository for 3D models, animation, plans, documents, all underlying a mission simulation
ECP Project Dust Mitigation (active) NNM05AA88C	Lunar dust mitigation strategies (Colorado School of Mines, DS)	Physics simulations of dust behavior in the lunar environment
ECP Project MODAT (active)	End to End Mission Simulation (ARC, USRA, DigitalSpace)	Creation of human-in-the-loop proofs of concept agent-driven environments: JPL mission ops
ECP Project SuperBots (active)	Innovative Robotics (USC, ISI, JPL)	Creation of visualization platform for SuperBots in multiple tasks
Raytheon CE&R (complete)	Lunar exploration architecture	Rapid prototyping of lunar reference architecture
Boeing CII (pending)	Space Exploration Online	Multi user game for outreach around the exploration vision

See Also:
Archive of DigitalSpace Proposals and other Publications.