Digital Space
 

FINAL TECHNICAL PROGRESS REPORT

For SBIR I: NNA05AC13C

Simulation-Based Lunar Telerobotics Design, Acquisition and Training Platform for Virtual Exploration

Report Date: July 18, 2005

Reporting Period: January 25, 2005 – July 18, 2005

PI/Contact: Bruce Damer, DigitalSpace Corporation

NASA Program Recipients

NASA Ames Research Center

Accounting Operations Branch, M/S 203-18

Moffett Field, CA 94035-1000

CS Amine Hambaba, MS 241-1

COTR Mark Shirley, MS 269-3

SBIR Specialist MS 202A-3

SBIR Team Recipients: B Damer (DigitalSpace), G Brandt (DigitalSpace)

Phase I Project Summary

Firm: DigitalSpace Corporation

Contract Number: NNA05AC13C 

Project Title: Simulation-Based Lunar Telerobotics Design, Acquisition and Training Platform for Virtual Exploration

Identification and Significance of Innovation:

To meet the objectives of returning to the moon by 2020 for extended stays leading to human exploration of Mars will require NASA to fly a series of telerobotic lunar orbital and surface explorers, some to prove the viability of In Situ Resource Utilization (ISRU) and others to engage in lunar exploration and human base precursor construction. For such a large number of vehicles and innovative designs, a real-time 3D tool that creates plausible simulations of space environments and vehicles can bring an urgently needed rapid prototyping capability to the new vision for space exploration. 

Digital Spaces (DSS) is a 3D rapid prototyping platform that will allow distributed teams to iterate designs on a week to week basis to craft operational scenarios with real-time 3D simulations tightly linked to team meetings, teleconferences, web documents, CAD databases and other decision support tools. DigitalSpace completed the first phase of this SBIR project to model the Colorado School of Mines’ (CSM) prototype lunar bucket wheel excavator (BWE) to demonstrate that a real-time simulation of a lunar surface vehicle is possible. The BWE was chosen as it is one of the only lunar ISRU-oriented prototypes in the world and is the lead vehicle in a series that is likely to include trenching, boring and hauling machines. The BWE simulation allows users to run real-time traverses of the vehicle on a virtual landscape populated by elements of a hypothetical lunar base. A force feedback joystick and stereoscopic immersive display is supported to create the operator sensation of a real excavator and yet the environment runs on ordinary consumer computers. It is hoped that this tool will evolve from a design simulator into a prototype operator interface, a mission trainer, and finally a tool for day to day surface operations.

Technical Objectives and Work Plan:

This project sought to create a simulator of an existing prototype lunar bucket wheel excavator (BWE), the testing of that virtual robotic vehicle inside a virtual lunar regolith stimulant sand box and the final application of the combined virtual environment with haptic force feedback devices and multi-modal  immersive displays. The work plan objectives were: 

Prototype Bucket Wheel Excavator

Create dynamic 3D virtual model of the existing (CSM) prototype lunar analog BWE guided by experts from the CSM. 

Virtual environments

Create a virtual simulant “sandbox” to test the virtual excavator based on approximate physical properties of the lunar regolith including gravity.
 

This Phase I proposal successfully developed and tested a software-based virtual test fixture performing a 3D dynamic simulation of a lunar bucket wheel excavator (BWE) prototype built at the Colorado School of Mines (CSM). This simulation can be run in real-time on most PC workstations and supports a physics simulation of the vehicle on a lunar surface environment. The following work plan objectives were met: 

Built 3D virtual model of the existing prototype lunar analog BWE and validated by experts from the Colorado School of Mines. 

Created a virtual stimulant sandbox to test the excavator model. The sand box was validated by CSM testing with commentary on future simulation improvements. 

A simple control interface using a force feedback joystick was developed to operate the virtual excavator in the sandbox. 

The full version of the simulation was output onto a quad buffer auto-stereo display to show that a sense of immersion can be given to the operator. 

The virtual prototype was iterated to match the properties observed by CSM and others in their original testing of the physical excavator in its simulant sand box. Guidance was obtained about future rapid prototypes of teleoperated lunar vehicles and missions.

NASA Application(s):

NASA applications enabled by a collaborative real-time 3D rapid prototype platform include: 

1. Design simulation of teleoperated surface robotic vehicles and missions on the Moon and Mars for exploration, site preparation and In-Situ Resource Utilization (shielding, materials, fuel and water extraction).

2. Concept development for teleoperated robotics for servicing for ISS, orbiting scientific observatories (Hubble) and long duration Crew Exploration Vehicles (CEV).

3. Simulation of astronauts and robotic agents within spacecraft or in surface facilities.

4. Telerobotics training for all classes of remote manipulator systems.

5. Training and mission operations using the virtual environments as an operator tool.

6. Design and prototyping of other types of vehicles through integrated modeling and simulation.

Non-NASA Commercial Application(s):

Any remotely controlled vehicle in mining, construction, hazardous waste handling, military operations and other commercial applications requires high fidelity virtual environments for development of viable designs, interaction scenarios, and eventually for the training of operators and day to day operations. The needs for teleoperations in the terrestrial mining, construction and manufacturing industries alone could create a multi-billion dollar annual business. We have identified the following applications in design simulation that could be served by the platform:

1. Collaborative engineering of automated mining especially deep mining where heat and hazards make ore extraction costly or prohibitive.

2. The design of automated construction including excavators, haulers, cherry pickers, and cranes.

3. Defense and security applications in the design, training and operations for teleoperated vehicles in the battlefield or for facilities surveillance.

4. Planning and training for emergency first responder hazardous area robotics.

5. Industrial design, training and operations applications.

6. Software games in the “robot wars” genre and for education/outreach in robotics education.

Name and Address of Principal Investigator: 

Bruce Damer, DigitalSpace Corporation, 343 Soquel Ave, #70, Santa Cruz CA 95062. 

Name and Address of Offeror: 

DigitalSpace Corporation, 343 Soquel Ave, #70, Santa Cruz CA 95062

Part 2 Identification and Significance of the Innovation

2.1 Identifying the Need

On January 14, 2004, President George W. Bush enunciated a new vision for space exploration [1,2] which set forth a road map to return humans to the moon by 2020 and set the stage for the human exploration of Mars. One year later, the first NASA Robotic & Human Lunar Exploration Roadmap Meeting was held in Houston and NASA’s Chief Scientist, Jim Garvin, and others presented strategies and options for robotic and human exploration of the moon (including 14 day stays at a prepared site) as a test bed for the missions to Mars. Dr. Garvin’s Lunar Strategy [3], was echoed by the other participants including the members of the newly formed Lunar Exploration Analysis Group (LEAG). In each scenario, after the Lunar Reconnaissance Orbiter (LRO) mission of 2008 has identified candidate sites, a series of teleoperated fixed landers and long range rovers will be deployed to the moon to engage in geologic exploration of the Lunar regolith and searching for water ice and other elements for possible In-Situ Resource Utilization (ISRU) in support of human lunar visits.

In the lunar exploration community there is a strong consensus that ISRU is necessary for sustainable (and affordable) human presence in space [4]. The LEAG recently identified the following priorities for early experimental test beds, research and flight missions [5]:

1. Learn how to move and handle regolith (excavation, drilling and coring, hauling, stacking, hopper-feeding).
2. Attempt the thermal processing of regolith (e.g., sinter to make pavement).
3. And to assess techniques for resource extraction from regolith (e.g., oxygen via reduction of regolith, extraction of ice for fuel and mission consumables).

A real-time 3D environment that creates plausible simulations of space environments and vehicles can bring an urgently needed rapid prototyping capability to any NASA mission and vehicle design effort. Going well beyond static artist’s conceptions, this prototyping tool will allow distributed teams to iterate designs on a week to week basis crafting operational scenarios with real-time 3D simulations tightly linked to team meetings and teleconferences, web documents, CAD databases and other decision support tools. DigitalSpace proposed and completed the first phase of this SBIR to model the Colorado School of Mines’ lunar bucket wheel excavator (BWE) to demonstrate that a real-time simulation of a lunar surface vehicle is possible (figures 1-4). The BWE was chosen as it is one of the only lunar ISRU-oriented prototypes in existence. The BWE is the lead vehicle of several classes of mining machines which include further design iterations into trenching and boring machines, all likely to be employed in lunar exploration and ISRU.
 

Fig 1: Dr. Michael Duke with Colorado School of Mines’ Lunar Bucket Wheel Excavator (R. King, T. Muff, M. Duke).	Fig 2: DigitalSpace’s reconstruction of the excavator as a virtual vehicle.
Fig 3: The excavator simulated climbing a radiation-mitigating regolith mound next to a partially buried lunar habitat.	Fig 4: Test of virtual lunar excavator at the Colorado School of Mines with consumer laptop and force feedback joystick interface (by Project Dust PI Masami Nakagawa).

2.2. The Innovation
 
 

Figure 5: Rapid prototype prepared for USC’s SuperBots NASA ECP project showing a superbot design at work on a hypothetical CEV EVA (June 2005)	Figure 6: Lunar precursor telerobotics base preparation visualization in support of Raytheon’s CE&R (January 2005)

During the six months of this phase I project, DigitalSpace was called upon separately to support rapid turn-around 3D visualization and telemetry generation for a number of NASA ECP (extramural) projects (USC/Superbots shown in figure 5), a NASA CE&R presentation for a major subcontractor (Raytheon shown in figure 6) and an Exploration Systems Mission Directorate (ESMD) prototyping project (Distributed Observer Network). During this period, Mark Shirley, the COTR for this phase I SBIR effort, identified that there is a need for a platform to present shared 3D real-time simulations which allowing recording, playback and annotation and can be run by in-person or teleconferenced participants on desktop or laptop computers. It was against this backdrop that we completed work on this Phase I SBIR and discerned a vital need that could be met during a possible second Phase of this work. 

Meeting ESMD’s collaborative rapid design prototyping needs

It is clear that over the next several years there will be a large number of mission trade studies, reference mission architectures, in-person conferences and teleconferences needed to support the implementation of the President’s new vision for space [1, 2]. Static images of mission hardware and scenarios or simple 3D animated movies have traditionally provided a starting storyboard for mission visualization but those media are too inflexible to support actual ongoing engineering design iterations and qualification of operational approaches. In addition, NASA and its contractor community will continue to rely heavily on voice conference calls backed up by email and document presentations. In near term, complex mission scenarios such as CEV flights, lunar telerobotic base construction and human lunar habitation, there will be increased challenges for team members to build effective mental models of overviews of proposed mission scenarios as well as detailed design options. We believe therefore that a gap exists between the static artist’s conceptions, teleconference notes and documents and the engineering commit stage of full computer aided design (CAD) and prototype construction (“bending metal”). It is in this gap where really clear understanding of complex vehicle/mission scenarios is so important that a rapidly iterated 3D prototyping environment can make a big difference. 

The new enabling technology of real-time 3D

It has been only in the past several years that ubiquitous and powerful 3D chipsets have become standard in virtually all PCs and laptops. Therefore, real time 3D rendering is possible, with the end user able to change elements of a simulation on the fly. These environments are moving into the engineering mainstream. As an early investor in much of the serious real-time 3D industrial applications [7], NASA has a great legacy to draw on for 3D in mission simulation, training and mission operations [8]. In the new era of exploration, NASA has a chance to again pioneer innovative uses of real-time 3D. In addition, in the era of open systems standards and open source, platforms can take advantage of tremendous economies of scale in development. 

With support by NASA, USRA and others since 2000, DigitalSpace has built an open platform called “Digital Spaces” (or DSS) which is customized for a range of NASA mission and knowledge sharing needs. The architecture for DSS is shown in figure 7 below. 

Figure 7: Digital Spaces (DSS) architecture, summer 2005

Bringing rapid prototyping to the design of space vehicles and mission scenarios

Virtually every field in which complex machines or parts are built on a short timeline utilizes some form of rapid prototyping. In manufacturing, expensive injection or press molds are machined only after iterations through 3D solid object “printers” which produce physical test articles out of low cost materials in time frames as low as a few minutes [9]. This step greatly reduces errors and improves design prior to the costly machining of final parts. This kind of rapid prototyping is a metaphor for the platform proposed here. Real-time 3D environments that create plausible simulations of space environments and vehicles will bring a rapid prototyping capability to any early stage mission and vehicle design effort. 

Proposed future platform features

As a result of this Phase I project, described in Part 3, we have derived the following feature set we believe would merge the world of rapid virtual prototyping with work practice norms at NASA.We plan to propose these in Phase II: 

1. Synchronized simulations for a given vehicle or mission scenario, design and engineering team members will run through the Internet as a supplement to normal teleconferences and physical meetings. 

2. Recording and playback of member-driven simulation or operations sequences: all simulation sequences (whether pre-programmed or driven dynamically by a team member) can be made available to project managers, engineers or participants who may have been absent for a given meeting. These recorded simulation sequences will also provide an important historical log of decisions made. 

3. Annotation of these simulation sequences would be possible by direct user interaction. Meeting notes, decisions and attached media such as images, audio and outside references could all be attached to the 3D models or within the interface and stored in the Universal Model Repository (UMR) now in development under another SBIR project at DigitalSpace (figure 7 above). In a 2004 SBIR phase I project for NASA ARC, DigitalSpace created an online version of the SimStation real-time visualization of the ISS (reference “Sim Station Online” project at [10, 11]) Selecting all or part of the real-time model executes a SQL query and brings up any media type, database record or document. Changes to annotations or associations would be shared with team members in real time. In between meetings, any project member 

Mission profiles served by the tool

NASA’s new exploration vision will require multiple classes of mission to move human explorers beyond Earth’s orbit. Since 1999 DigitalSpace has carried out a range of projects with NASA, its contractors and others which we believe puts us in a strong position to create a key tool to support multiple mission collaborative design needs. We will next examine how this tool will serve critical collaborative design needs for the following mission profiles: 

· Surface exploration and ISRU validation missions

· Crew Exploration Vehicle

· Telerobotic and Human base operations

Surface exploration and ISRU validation missions
 

Figure 8: Simulation of a sintering ISRU excavator design for Raytheon/Colorado School of Mines.	Figure 9: Colorado School of Mines’ lunar bucket wheel excavator simulation (this Phase I project).

Early lunar missions after Lunar Reconnaissance Orbiter (LRO) flies in 2008 may include MER-style long duration surface resource explorers seeking deposits of water ice and other materials to lower mission costs and risks. The rapid prototyping of one or more designs for these vehicles will be a primary focus of this tool. The phase I project was directed mainly at this goal by modeling the lunar bucket wheel excavator (figure 9) and related work on sintering (figure 8). 

Crew Exploration Vehicle (CEV)
 

Figure 10: A rapid real-time 3D visualization of a hypothetical Lunar CEV design DigitalSpace produced for Boeing, Spring 2004.	Figure 11: Simulated CMG changeout for STS-114 & RPCM changeout for ISS (Raytheon and NBL)

The CEV and its associated heavy lift cargo capability possibly based on Shuttle derived technology will bring crews and mission hardware to LEO and beyond in the post Shuttle 2010 time frame. DigitalSpace produced 3D visualizations of CEV for Boeing and Raytheon in 2004 (figure 10) and this tool should be well suited to rapidly rendering designs for CEV in ascent, in-orbit, docking and undocking, EVA and entry, descent and landing. A project for Raytheon and the program office of the Neutral Buoyancy Laboratory at JSC (figure 11) allowed us to experience creating a training simulator for crews in preparation for Shuttle/Station missions (in this case STS-114 return to flight). The ability to model IVA or EVA aboard the CEV will be a key element to design of both vehicle and missions. 

Telerobotic and human base operations
 

Figure 12: EVA during Mobile Agents field test, Mars Desert Resarch Station, Hanksville UT, April 2005	Figure 13: Planning meeting simulation from 2002 BrahmsVE project to model a day in the life of the FMARS analogue Mars habitat, 2002

Figure 14: Plan view of Lunar CEV crew EVA simulation for the BrahmsVE project, Summer 2005	Figure 15: Annotated simulation of Lunar CEV crew EVA to check base for a stay/go decision, Summer 2005

DigitalSpace’s prior work supporting the Brahms team at ARC and USRA for simulations of activities during their FMARS and MDRS analogue Mars base field seasons [12, 13, 14] (figures 12, 13) allowed us to get an early start in modeling human/robotics and the current MODAT ECP project puts it in an excellent position to support complex human-in-the loop human/robotics mission profiles. The parallel BrahmsVE project is now bringing Brahms agents to a lunar base simulation (figures 14, 15). 

More on the recommendations from the testing of this Phase I work product and for future development tasks in part 7.2.

Part 3 Technical Objectives

3.1 The Objective

The solicitation for this topic called for innovation in the way that people will interact with both physical and virtual data sets using multi-sensory displays and interfaces (including force-feedback) to support richly endowed situational awareness and telerobotics.

Final work product: Simulation of the BWE in an analogue lunar base environment
 

Figure 16: The CSM’s Lunar Bucket Wheel Excavator concept vehicle	Figure 17: Bucket wheel operating in testbed stimulant

The prototype Lunar Bucket Wheel Excavator (BWE) [15] concept vehicle (figure 16) was developed at the Colorado School of Mines under the direction of Dr. Michael Duke, Dr. Bob King and was built by graduate student Tim Muff (now at Lockheed-Martin Corporation). This vehicle was tested in a Lunar regolith stimulant sandbox (figure 17) and its ability to transport (mine) stimulant was characterized (see reference 1 in section 6 below). DigitalSpace selected the Lunar BWE as the focus of this Phase I SBIR for the following reasons:

The CSM’s BWE is one reference design among many possible and represents one very early data point. The CSM’s lunar stimulant sandbox, based on but not using the stimulant JSC-1, is only a very rough analogue of lunar soils. In addition, the lunar environment of a near vacuum, one sixth of Earth’s gravity and the thermal and radiation environment, was not simulated for the CSM’s sandbox testing. Lastly the communications latency present with lunar teleoperations was not simulated in the CSM testing nor were operator interfaces such as haptic joystick controls typically found in terrestrial mining.

Despite these limitations, the CSM’s BWE is the only existing example of a concept lunar construction robot and serves as an excellent starting point for early research into mission-critical lunar resources utilization and telerobotic operations. The goal of this SBIR Phase I was to build on the strengths of this work to produce a virtual analog of the BWE in the lunar environment as a test fixture for the evolution of lunar telerobotic systems that may lead to flight hardware. The goals of the SBIR project are:

1. Produce a virtual model of the lunar BWE concept vehicle in as close detail as possible, with expert advisement from the vehicle’s designers.
2. Place this virtual model within a synthetic lunar analogue environment, constructed as a 3D model, physics-enabled, simulated Lunar gravity and rudimentary surface characteristics (no radiation or thermal environment modeling will be attempted).
3. Create an operator control modality for the virtual vehicle through the use of force feedback (haptic) joystick interface to drive the vehicle. Simulate machine-soil contact through the use of a physics engine and tie the computed contacts to the joystick to allow an operator a “heavy equipment operator” type of experience familiar to mining and terrestrial equipment operators and designers. Introduce the 2.6 second signal communications latency that would be experienced by future lunar teleoperators.
4. Test and iterate this virtual test fixture environment with experts in mining, the lunar environment and virtual vehicles/simulators. 

New additions made to the Digital Spaces platform
 

Figure 18: Current Digital Spaces architecture showing physics engine and haptic device extensions

Digital Spaces (DSS) is organized like an operating system and consists of three elements, core modules, akin to kernel elements in an operating system such as Linux, plug-ins which are extensions connected to the Core and other plug-ins by an abstracting API, and libraries, which may be databases of objects or groupings of functions. Figure 18 above illustrates the significant additions of modules, plug-ins and libraries during this phase I project: 

1. A physics module which through an additional library called Gangsta, permits a wide range of open source and proprietary physics engines to drive physics in a common scenegraph. 

2. A module for a force-feedback joystick (haptic device). 

3. Enhanced full screen 3D rendering of the simulation with good performance and overlaid 2D graphical interface objects (Ogre renderer and CEGUI library). The renderer also supports immersive autostereo displays through NVidia quad-buffer hardware. 

4. A dedicated BWE dynamics simulator plug-in to match the behavior  of the physical BWE as described by its designer, Dr. Robert King. 

5. Additional modules such as a new Camera Control, Entity State Display and Animation sequencers and playback elements were added. In the same period, a plug-in for the Brahms agent system was completed for a separate project. Brahms will be an important element of a Phase II proposal of this work.

The unique achievement of a multi-engine physics implementation

DigitalSpace determined that for any surface vehicle simulation, physics simulation would play a key part. As there are a dozen open source and proprietary physics engines available on the market, each having differing strengths and weaknesses, we determined that it would be unwise to create a dependency on only one solution. We hired a physics specialist, Ed Jones, who built what we believe is the industry’s first multi-engine physics module. Ed created and tested an interface (called Gansta) to four of the leading engines (ODE, Novodex, TrueAxis and Newton) which is now integrated into DSS. Figures 19-22 below illustrate our testing of three of the four physics engines via this plug-in methodology. The before and after simulations (tower of objects falling, robot arms lifting boxes) show the slightly different performance of the three engines running in real time in the same scenegraph. The final work product uses the open source ODE physics engine.
 

Figures 19-22: three physics engines in before-after scenario testing via the generic DSS plugin

Tying a generic game force feedback joystick into a 3D scenegraph with physics

Another significant achievement of this 6 month effort is that we were able to tie the generic DirectX support for low cost game joysticks having a force feedback (haptic) capability into both the DSS 3D scenegraph and the physics engine. Figure 23 illustrates how these modules interact to drive physics by joystick or glove and output on an autostereo display (figure 24).
 

Figure 23: DSS architecture extensions illustrating feedback between physics engines and force feedback joystick (3D Input Module).	Figure 24: virtual BWE on autostereo display with immersive shutter glasses.

Delivery and test/evaluation schedule

The work product was delivered in stages by DM3D Studios (DigitalSpace’s subcontractor on the project) for alpha testing internally at DigitalSpace in March 2005 and the first on-site evaluation at Colorado School of Mines occurred during a site visit in April 2005 (see figure 5 above). There was a test and public demonstration given in Melbourne, Australia in May 2005 run for a week as part of an exhibit on virtual environments in the future of manufacturing sponsored by Swinburne University. Following the addition of the physics and force feedback and beta testing at DigitalSpace in June, the whole package was made available for test in early July, 2005 to Colorado School of Mines, NASA ARC, andStanford University.

Reasoning for not integrating Stanford Biocomputation’s Spring system

We opted (after discussing this with Dr. Kevin Montgomery of the Stanford Biocomputation Center) to not integrate their Spring system and hardware for reasons of cost, time and availability. We agreed that supporting standard force feedback game joysticks such as the Logitiech® Force3D Wingman within the standard Windows D3D driver would give us a good approximation (initial experience) with haptic control. In the course of the project three of these devices were acquired and shipped to developers and testers.

Final work product package

In conclusion, the work product delivered included: 

1. Open source DSS code base additions in Content Management/Versioning System and our open physics solution was made available to a wider community on SourceForge. The entire code base is available at www.digitalspaces.org
2. Users Guide and FAQ
3. Joystick (Logitech Force Feedback 3D wingman model) and drivers when requested
4. BWE model and content of analog lunar surface base available to render as a stand-alone application
5. Operating system support was implemented under Windows 2000/XP using DirectX 7, 9, or  OpenGL with the option of using NVidia quad-buffer autostereo projector devices (see figure 24 above).
6. Secure site to download installation routines, DSS core and extensions, 3D models and documentation. Total disk space requirements after installation: 12MB

As the figures 25-32 below illustrates, the interface to the Digital Spaces’ lunar bucket wheel simulation permits a number of operating modalities: 1) camera tracking of the virtual vehicle from an independent viewpoint or tied to the vehicle motion, 2) operation of the simulation through a joystick or keyboard-based interface, 3) force feedback sensing in the joystick interfaces depending on the machine-regolith contact of the wheels/chassis of the virtual vehicle and of the bucket wheel contact with the simulated regolith, 4) teleoperation simulating the 2.6 second latency representing the communications time from Earth to moon and return (direct). In addition, the position and orientation of the virtual vehicle is reported. 
 

Figure 25: the BWE scaling a landing exhaust berm protecting the habitat in DigitalSpace’s test lunar base environment.

Figure 26: BWE in drive mode over simulated regolith.	Figure 27: BWE bucket wheel “biting in” with simulation of dust effects.

We used a simple “puff” of dust to indicate contact of the bucket wheel with the simulated surface. Colorado School of Mines has been awarded a Phase I ECP project to examine dust mitigation strategies for the moon and Mars and has agreed to participate with us in a Phase II SBIR proposal. Part of this participation will allow us to produce particle simulations of likely dust conditions in the lunar environment. These simulations, associated with lunar vehicles, will aid “Project Dust” at the CSM to study mitigation strategies.
 

Figure 28: Detail view of BWE bucket wheel and arm mechanism along with screw for drawing material up through the vehicle.	Figure 29: BWE caught on hilltop showing physics action of double rocker-bogie mechanism. Vehicle is able to back out of such situations.

Figure 30: BWE backing up successfully to hopper on ISRU processor.	Figure 31: BWE off normal and colliding with support stand for ISRU processor.

Figure 32: View from BWE to lunar base, with “track camera to vehicle” viewpoint

Reports on the testing of the BWE simulation

As this was a proof of concept project there was no statistical testing done. The Colorado School of Mines (CSM) was the main center for test although the COTR from NASA ARC also tested the simulation. The CSM testers were impressed by the work product and tested for several hours over a weeklong period. At the time of writing of this report they were still testing with several faculty. They felt we had captured the dynamics of the vehicle accurately and that it behaved as the physical robot does in its sandbox in terms of: a) the rocker-bogie mechanism was a plausible approximation, b) the behavior of the bucket wheel itself and its “bite in” felt realistic in the haptic interface, c) the felt that having the base elements such as the hab, blast ridge, and ISRU processor made for a valuable, more realistic test environment.

Where improvements could be made were:

1. A nonlinear interface from the joystick to the virtual vehicle drive (ie: the more the stick is pushed the faster the vehicle should driver) would be a better approximation.

2. The bucket wheel itself should move about a half to quarter of its current speed.

3. The bucket wheel should be able to be operated in “sweep” mode (moving side to side as normal mining BWEs do).

4. The first simulation of lunar regolith should be that if the bucket wheel digs in deeper it should be driven harder (resistance) and cause the vehicle to push back.

5. Collisions with base objects, such as the ISRU processor, should give some measure of whether the event damaged any equipment (metal stress points etc).

6. Simulation of dust should be more realistic (not “puffs” but ballistic arches) and that dust cannot “blow away in a breeze” as there is no atmosphere on the move to drive it but simply falls. The BWE wheels should also generate dust.

7. Simulation of regolith properties such as density, rock objects and material flows would be nice (although they pointed out how difficult a challenge this is).

The CSM testers and evaluators were very encouraging and looked forward to an opportunity to iterate the simulation again. We felt that this was a major endorsement of the concept of rapid virtual prototyping and forms the basis of our Phase II proposal.

Some of the results of the NASA ARC testing are as follows: 

1. It would be very useful to have a pattern on wheel sidewalls to be able to determine if we are in motion (and a distinct shape on top of wheels per Mike Sims to match how they track MER's progress visually). 

2. Collision and other geometry should be visible in a wire frame view mode. 

3. We must improve the joystick interface (providing a definable template ideally) to allow the user to navigate their own point of view with the joystick while the rover drives via keyboard or somehow merge these modes. The SimStation modality (derived from Descent) is a good starting point for a better joystick control. 

4. The joystick interface and haptic (force feedback) feedback was crude at best, given the use of sub-$100 game device technology. Due to the cost and custom programming interfaces, we opted not to use higher-end haptic devices such as those utilized at Stanford’s Biocomputation center. Testing with higher caliber human-machine interface devices in a subsequent phase of development could involve technology already in use for mining teleoperations [16, 17].

We believe that the Phase I project has proven the feasibility of producing a plausible real-time 3D visual simulation of a lunar surface vehicle and mission scenario in much higher fidelity. We are confident that DigitalSpace’s prior experience in building collaborative and multi-user platforms will enable us to create DSS-Prototyper, a marriage of the 3D rapid prototype simulation and multi-user collaborative tools (reference Phase II proposal).

See part 7.2 below for suggestions of future development made by the tester/evaluator teams.

Part 4 Work Plan

4.1       Technical Approach

The project commenced with extensive consultation with expert advisors at CSM/CCACS (the Center for Applications of Combustion in Space, now called the Institute for Space Resources (ISR), Stanford, and several NASA centers as other contractors and university collaborators. Thereafter the following staged implementation was be undertaken: 

· Produce a 3D virtual model of the existing ISR prototype lunar analog bucket wheel excavator. Validate the virtual excavator with the ISR. 

· Create a virtual stimulant “sandbox” to be able to test the virtual excavator. Endow the virtual sandbox with basic physical properties. Validate sand box with the ISR. 

· Build a simple control interface and test operating the virtual excavator in the sandbox. Match the control interface to likely haptic force feedback controls. 

·  Iterate the virtual prototype to match the properties observed by ISR in their original testing of the physical excavator in its simulant sand box. 

· Export the refined virtual model to the Spring system and work with Stanford Biocomputation center to create a first generation haptic force feedback version of the virtual excavator and stimulant sandbox. 

· Test the haptic-enabled version in an immersive display interface (parabolic) and generate data/metrics to compare simulator/haptic and observed physical performance. 

· Produce reports and recommendation for Phase II. 

Throughout the second and third portions of the project, all consulted teams were be able to view progress and the final simulation through releases of the DSS platform and project content. 

4.2       Task Descriptions

Table 1 below maps our allocation of hours by labor category for the eleven major tasks in this Phase I work plan and corresponded to our budget. 

Table 1: Work Plan
 

Total of all hours: 890 where the roles are defined as:
 

4.3       Meeting the Technical Objectives
 

All of the technical objectives in this Phase I project were met. 

Part 5 Potential Applications

5.1 Potential NASA Commercial Applications

Surface operations on the Moon or Mars, teleoperated servicing robotics on long duration space station and crew exploration vehicle flights and a whole class of crew training and data rich control environments demand a new generation of haptically enhanced virtual environments. Key NASA applications enabled by an advanced closed loop virtual environment telerobotics simulator include:

• Teleoperated surface operations on the Moon, Mars for exploration, site preparation and In-Situ Resource Utilization (shielding, materials, fuel and water extraction).
• Teleoperated servicing for ISS, orbiting scientific observatories (such as Hubble) and long duration Crew Exploration Vehicles (CEV).
• Teleoperated robotics for IVA (such as ISS/PSA)
• Teleoperation for enhanced utility and safety for launch facilities.
• Telerobotics training for all classes of remote manipulator systems.
• Design and prototyping of many types of vehicle through the use of the simulator for simulation-based design and acquisition.

Chemical, biological or nuclear hazards will all require first responders to consider using teleoperated machines to enter affected areas first. This work could directly inform this type of technology for occupational safety.

Online games – educational and entertainment applications

Experience:                   Mr. Damer is the world's recognized expert on avatars and shared online graphical virtual spaces having created much of the early literature, conferences and awareness of the medium. Mr. Damer is a visiting scholar at the University of Washington HIT Lab and a member of the staff at the San Francisco State Multimedia Studies Program. 

Name:                           Galen Brandt (Program Management, Marketing)

Position:                        New business development, DigitalSpace 

Experience:                  27 years including project management and marketing for Dun and Bradstreet, SUNY Fashion Institute of Technology, DoToLearn. 

Assignment:                  Program management including in reviewing all aspects of the work product and documentation. 

Name:                           Dave Rassmussen (SE)

Position:                       Principal of DM3D Studios, subcontractor to this project

Experience:                  10 years experience in virtual world design, skills: 3DS Max, Java, Active

Worlds, Adobe Atmosphere, PHP/MySQL database development

Assignment:                 Dave has served as senior engineer for this phase, directing the team performing 3D modeling and animation, testing and delivery of the project 

Name:                           Merryn Nielsen (Lead CD)

Position:                       Partner, DM3D Studios

Experience:                  10 years experience in virtual world design and web page creation. 

Assignment:                  Web & graphic design on project, 3D worlds, administrative support. 

Name:                          Peter Newman (TE)

Position:                       DM3DS 5 years experience in C++, JS, PHP, HTML 

Assignment:                  Chief technical architect of Digital Spaces (DSS) core and extensions. 

Name:                          Ryan Norkus (CD & TG)

Position:                       DM3DS 5 years experience Graphic artist, 3d modeler and animator 

Assignment:                  Focusing on 3D model construction & animated sequences 

Name:                           Peter Meigs, Alex Grigny de Castro, Dean Strik (TE)

Position:                       Server support and infrastructure, DigitalSpace 

Assignment:                  Focusing on the maintenance of DigitalSpace’s server infrastructure

6.2 Key NASA Participants

Key NASA participants in this project have included: project COTR, a number of NASA personnel at Ames Research Center including Michael Sims and others (see below).

6.3 NASA and Non-NASA Advisors and Evaluators

To facilitate both the live testing of the simulation of the lunar bucket wheel excavator, DigitalSpace recruited a team of individuals representing some of the world’s top expertise in robotics, the lunar environment and planetary surface exploration. 

Table 3: Expert advisor/evaluator team participants in the project
 

Brad Blair	Robert King	Gary “Rod” Rodriguez

Masami Nakagawa	Brahms/MODAT/Viz team

Brad Blair, of the Institute for Space Resources (ISR, formerly CCACS) at the Colorado School of Mines, is an internationally acknowledged expert in all aspects of lunar resources including the physical challenges of regolith handling and processing, the economics of ISRU and how it impacts whole mission architectures. Brad brought the depth of his knowledge of the space resources topic and community to bear on the project. Brad will also be assisting in directions for Phase II development and Phase III commercialization in the mining industry. 

Dr. Robert King is Full Professor, Division of Engineering, Colorado School of Mines, and the lead designer of the lunar bucket wheel excavator working with Mike Duke and graduate student Tim Muff. Bob’s focus is all matters involving the dynamics and operability of machine/earth contact. Coming in with a focus on mining engineering, Bob applied his background to the BWE’s anticipated mission in a lunar setting. Bob was also involved in testing the Phase I deliverable.. 

Gary “Rod” Rodriguez, President of SysRAND, member of the ISR and Program Manager of “Project Dust” at the Colorado School of Mines brings years of experience of system design and ISRU background to the project. Rod brought his experience to bear during testing of the virtual BWE. 

Dr. Masami Nakagawa, a professor at the Colorado School of Mines, brings his world-renowned lab and experience in dust and granular materials to the project. Working with Rod as PI on Project Dust, a NASA funded ECP efforts, he was one of the first testers of the environment during our site visit in April 2005 (figure 4). 

Brahms/MODAT/Viz teams reviewed the Phase I deliverable and commented on the application of Brahms agents and Mobile Agents methodologies to future versions. In addition, the lunar base environment in which the BWE is being tested was developed as a BrahmsVE application for this team. 

NASA ARC COTR Mark Shirley, and ARC/MER Science Co-Investigator Michael Sims both reviewed the platform. 

Part 7 Technical Activities

7.1 CumulativeTechnical Activities

This Phase I proposal successfully developed and tested a software-based virtual test fixture performing a 3D dynamic simulation of a lunar bucket wheel excavator (BWE) prototype built at the Colorado School of Mines. This simulation can be run in real-time on most PC workstations and supports a physics simulation of the vehicle on a lunar surface environment. The following work plan objectives were met: 

· Built 3D virtual model of the existing prototype lunar analog BWE and validated by experts from the Colorado School of Mines. 

· Created a virtual stimulant sandbox to test the excavator model. The sand box was validated by CSM/ISR  testing with commentary on future simulation improvements. 

· A simple control interface using a force feedback joystick was developed to operate the virtual excavator in the sandbox. 

· The full version of the simulation was output onto a quad buffer auto-stereo display to show that a sense of immersion can be given to the operator. 

· The virtual prototype was iterated to match the properties observed by ISR in their original testing of the physical excavator in its simulant sand box. Guidance was obtained about future rapid prototypes of teleoperated lunar vehicles and missions. 

7.2 Future Technical Activities

Reviewers identified the following immediate improvements in the BWE simulation and gave us a road map to the future applications of this kind of platform.

Immediate task 1: Improve user interaction modality with joystick, and keyboard

Immediate task 2: Improve rendering views of the 3D scene

Testers suggested that a more comprehensive simulation of the lunar regolith, soil mechanics and density, handling characteristics and modeling, and better dynamics (power train, physical behavior) of the BWE be modeled. 

Future task 6: Addition of a scripting language

It was suggested to support the integration of SCHEME/Gambit to enable more flexibility in the platform and lessen the need to make programmatic additions for specific applications. 

Future task 7: Open content/architecture

Pas experience guided us to create a plug-in architecture and open source core. A key indication of success will be when another team other than DigitalSpace takes up the tools and builds their own content. Two universities have now taken DSS in-house for projects. 

Many of these future tasks will be addressed in the Phase II proposal in which we will detail the opportunity and benefits of a “DSS-Prototyper” platform. 

Part 8 Potential Customer and Commercialization Activities

In anticipation of Phase III commercialization and Phase II pre-launch marketing and customer activity, we have identified the following classes of valuable applications for DSS and the components it encompasses. Companies in these fields include Multigen, Sense8, Virtualis, Planet9, RealSense, and CommonPoint. 

8.1 Recent NASA Potential and Active Customer Activities

Project Dust at CSM

This ECP project has as its goals the simulation of dust conditions for tools and vehicles on the moon and will be using DigitalSpace’s platforms for that purpose in year two (2006). DigitalSpace is a member of this project. 

MODAT/MDRS and Mobile Agents

This current ECP project at NASA ARC is planning to utilize the new capabilities in a model of a lunar base and JPL’s mission operations for MER. DigitalSpace is a team member of this project. 

SuperBots project

This ECP project is using DigitalSpace to visualize simulations of innovative modular robots on the lunar surface, in CEV and other tasks. The work product of this Phase I is key to the success of that effort as it relies on physics simulation. 

ESMD collaborative visualization

DigitalSpace was invited to participate in the Distributed Observer Network (DON) project by KSC and Valador and we provided 3D models and telemetry. We expect that, as explained in Part 2.1 above, there will be a large need for collaborative rapid prototyping for a whole series of missions in the short term. DSS’ open source zero cost license should help many parts of NASA and its contractors adopt the platform. DigitalSpace and others hope to provide services and applications for this common open platform. 

LEAG lunar exploration prototyping tools

The Lunar Exploration Analysis Group (LEAG) has invited DigitalSpace to present DSS and the BWE application at the October 2005 conference in Houston, TX. We will present this work in a plenary and it will be seen by the largest collection of lunar exploration and ISRU experts ever assembled. We hope it will become a tool for the LEAG. 

SimEVA and SimIVA

Recent applications for the Neutral Buoyancy Laboratory at JSC and the VAST project for medical emergency training aboard ISS suggests there are a number of immediate applications for a real time rapid prototyping virtual environment in training and mission planning. We anticipate building Haptic and other device interfaces through partnerships built for the VAST project.

NASA educational outreach and Space Camp
Immersive virtual worlds, virtual digital human (VDH), and 3D simulation modeling, have become a significant vehicle for NASA's effort to generate and communicate knowledge/understanding to K-12 and college/university students on topics such as the International Space Station the new vision for moon/Mars exploration. DSS has been proposed for use at NASA Space camp to Ed Buckbee, founder of Space Camp. In addition, the Space Exploration Online massively multiplayer online game concept has been proposed to NASA headquarters education/outreach together with Boeing who have agreed to support the effort to develop a prototype in DSS. 

8.2 Recent Non-NASA Potential Commercial Activities

K-12 and College, Education and Educational Web Services

The current set of DSS applications is being repurposed into educational course modules for schools, museums and other organizations. In discussions with the organizers of National Space Week, the Planetary Society, the Mars Society, and SPACE.COM, various modules of DSS projects have been made available and are now features in DVD, web and installation projects. 

8.3 Other Recent Commercialization Activities

Public health applications

The National Institutes of Health, through the work of Dr. Dorothy Strickland and Do2Learn have contracted DigitalSpace to employ DSS to deliver safety games for children with autism. The first game was delivered for clinical trials at Emory University in Atlanta in July of 2004 and more work in this area is anticipated. 

8.4 Future Potential Customer and Commercialization Activities

Hotel design

Space architect Constance Adams has invited DigitalSpace to use DSS for a September 2004 presentation of new designs for hotel rooms which embody principles learned while designing optimal work and living spaces for ISS.

Robot games – educational and entertainment applications

Up to the report date 100% of the work has been completed. 

Part 10 References

[1] Office of Exploration Systems on the web at: http://exploration.nasa.gov and the President’s announcement of the new vision for space exploration on Jan 14, 2004 is at: http://www.nasa.gov/missions/solarsystem/bush_vision.html

[2] President's Commission on Moon, Mars and beyond, report on the web at: http://govinfo.library.unt.edu/moontomars/

[3] Garvin, Jim, “Strawman Robotic Architectures, Lunar Architectures” presented Jan 25, 2005 at the NASA Robotic & Human Lunar Exploration Roadmap Meeting 1, January 24-25, 2005, Space Center Houston. On the web at: http://www.moontoday.net/news/viewsr.html?pid=15540

[4] Space Resources Roundtable, LEAG and Lunar and Planetary Institute resources on the web at:

http://www.mines.edu/research/srr/ and http://www.lpi.usra.edu/meetings/leag2005/ and http://www.lpi.usra.edu/lunar_resources/

[5] G. Jeffrey Taylor, University of Hawai`I, Lunar Exploration Analysis Group (LEAG), presentation at the NASA Robotic & Human Lunar Exploration Roadmap Meeting 1, January 24-25, 2005, Space Center Houston. On the web at: http://www.hq.nasa.gov/office/apio/pdf/moon/05_leag_debrief_taylor.pdf

[6] LEAG contributions to NASA’s 60 Day Study from communication with Jeff Taylor, Jul 12, 2005.

[7] Multigen Paradigm library of projects on the web at: http://www.multigen.com/solutions/success.shtml

[8] R.B. Loftin and P.J. Kenney, Training the Hubble Space Telescope Flight Team, IEEE Computer Graphics and Applications, vol. 15, no. 5, pp. 31-37, Sep, 1995.

[9] Rapid Prototyping resources from the University of Utah on the web at: http://www.cc.utah.edu/~asn8200/rapid.html

[10] DigitalSpace projects and publications on the web at:

http://www.digitalspace.com and publications at http://www.digitalspace.com/papers

[11] Shirley, M., Cochrane, T., SimStation: A Knowledge-Integrating Virtual Vehicle, Virtual Iron Bird Workshop, NASA Ames Research Center, March 31, 2004. Available on the web at:http://ic.arc.nasa.gov/vib/day1/papers/Shirley_Cochrane.pdf

[12] M. Sierhuis, Modeling and Simulating Work Practice; Brahms: A multiagent modeling and simulation language for work system analysis and design, Ph.D. thesis, Social Science and Informatics (SWI), University of Amsterdam, SIKS Dissertation Series No. 2001-10, Amsterdam, The Netherlands, ISBN 90-6464-849-2.

[13] B. F. Damer, et al, Final Report, SBIR I: BrahmsVE: Platform for Design and Test of Large Scale Multi-Agent Human-Centric Mission Concepts, DigitalSpace Documents, July 2004. On the web at: http://www.digitalspace.com/reports/

[14] Mobile Agents and Brahms publications on the web at: http://homepage.mac.com/wjclancey/~WJClancey/WJCCogSci.html

[15] Muff, T., Johnson, L., King, R., and Duke, M.B., (2004), A Prototype Bucket Wheel Excavator for the Moon, Mars and Phobos, Proceedings of the 2004 Space Technology and Applications International Forum (STAIF-2004), Albuquerque, New Mexico, February 8-11, 2004.

[16] Automated Mining resources and publications on the web at: http://www.automatedmining.ca/news.html

[17] Victor Mallet, Miners are drilling to unimagined depths in the search for gold, Financial Times Weekend June 5 / June 6 1999

[18] S. Trentanelli, Armed Robots Provide Glimpse of Future Warfare, RDECOM Magazine, Sept 2003, on the web at: www.rdecom.army.mil/rdemagazine/200309/itf_armed_robots.html

[19] Sony’s Aibo and iRobot’s Roomba  web at: www.sony.net/Products/aibo and www.irobot.com
 
 
 

(cc) 1995-2007 DigitalSpace, Rights are Granted for use of this material through our Creative Commons License.
Feel free to reach us through our contact form