U.S. Army Aviation and Missile Command

Acquisition Center

Aviation & Missile RDEC Directorate

“A Feasibility Study using Intelligent Software Agent Technology for Weapon-Target Pairing, utilizing Unmanned Aerial Vehicles”

Scientific and Technical Report

The EPV Group, LLC

671 S Mesa Hills

El Paso, Texas 79912

In Collaboration with

CDM Technologies Inc.

San Luis Obispo, California

CDRL# DI-MISC-80711A: Scientific and Technical Report

Intelligent Software Agent Technology for Unmanned Aircraft Systems (UAS)

A Feasibility Study

Intelligent Software Agent Technology for Unmanned Aircraft Systems (UAS)

A Feasibility Study

Table of Contents

Executive Summary …………………………………………………………………….. 1

Table of Contents ………………………………………………………………………. 3

1. Introduction: Intelligent Software as a Paradigm Shift ………………………… 5

1.1 Purpose and Structure of Report …………………………………………….. 6

2. Background: Unmanned Aircraft Systems ……………………………………… 7

2.1 Definitions ……………………………………………………………………. 7

2.2 Primary Military Applications ………………………………………………. 8

2.3 Technical Challenges Identified by DoD ……………………………………. 8

2.4 Analysis of DoD Findings: Overlooked Technology Opportunities …………. 11

2.4.1 Existing Intelligent Software in Military Domains …………………… 12

2.4.2 AI Application Opportunities for Unmanned Vehicles ………………. 13

2.5 The Need to Couple Unmanned Vehicles with AI Technology ……………… 14

3. AI Technology Underpinnings …………………………………………………… 17

3.1 Information-Centric vs. Data-Centric ………………………………………… 17

3.2 Service-Oriented Architecture (SOA) ……………………………………….. 18

3.3 SOA Implementation Components …………………………………………… 19

3.3.1 Services Management Framework (SMF) ……………………………. 19

3.3.2 Enterprise Service Bus (ESB) ………………………………………… 20

3.3.3 What are Services? ………………………………………………….. 22

3.3.4 Typical Service Requester and Service Provider Scenario …………… 22

3.4 The Need for Information-Centric Software ………………………………… 23

3.5 Layered Architecture …………………………………………………………. 24

4. Intelligent Software Agents ……………………………………………………… 27

4.1 Software Agents as Intelligent Decision-Assistance Tools …………………… 28

4.2 Planning and Re-Planning Functions …………………………………………. 31

4.3 Truth Maintenance Approach to re-Planning ………………………………… 32

4.4 Learning as Part of a Planning System ……………………………………….. 32

4.5 Service Agents ………………………………………………………………… 32

4.6 Mentor Agents ……………………………………………………………….. 33

5. Web Services ……………………………………………………………………… 35

5.1 Web Services as an Implementation of the SOA Concept …………………… 35

5.2 Simple Object Access Protocol (SOAP) …………………………………….. 35

5.3 Universal Description, Discovery and Integration (UDDI) ………………….. 36

5.4 Web Services Description Language (WSDL) ……………………………….. 36

5.5 Federated Web Services ……………………………………………………… 36

5.6 Web Services Security ………………………………………………………. 36

6. Interoperability Bridges …………………………………………………………. 39

6.1 Problem Definition …………………………………………………………… 39

6.2 Solution Criteria ……………………………………………………………… 40

6.3 Technologies Employed ……………………………………………………… 41

7. UAV Image Interpretation: An Example Demonstration ……………………… 43

7.1 Object Tracking ……………………………………………………………….. 43

7.2 The Adaboost Algorithm …………………………………………………….. 43

7.3 Online Adaboost ……………………………………………………………… 44

7.4 Features for Tracking ………………………………………………………… 45

7.5 Tracking as Classification ……………………………………………………. 46

7.6 Detecting the Object of Interest ……………………………………………… 47

7.7 Applications to Unmanned Aerial Vehicles ………………………………….. 47

8. Phase II Proposal …………………………………………………………………. 49

8.1 Objectives of the UVORIS Proof-of-Concept System ……………………….. 49

8.2 Technical Approach …………………………………………………………… 50

8.2.1 Case-Based Reasoning ……………………………………………….. 50

8.2.2 Ontology-Based Representation ……………………………………… 50

8.2.3 Rule-Based Agents ……………………………………………………. 51

8.2.4 Statistical Object Recognition Methodologies ……………………….. 52

8.2.5 Semantic Database Search Techniques ………………………………. 52

8.2.6 Knowledge Management Enterprise Services (KMES ®) ……………… 54

8.3 Phase II Task and Deliverables ……………………………………………….. 55

8.4 Estimated Phase II Duration and Cost ………………………………………… 57

9. Acronyms and Definition of Terms ………………………………………………. 59

10. References and Bibliography …………………………………………………….. 67

Executive Summary

This report explores the feasibility of applying artificial intelligence (AI) technology and in particular intelligent software agent methodologies to facilitate the remote guidance and improve the autonomous control capabilities of unmanned aerial vehicles (UAV). In their primary military applications of surveillance, target identification and designation, counter mine warfare, and reconnaissance, the effective deployment of UAVs is currently limited by inadequate software support. The two most critical software needs are cited in the literature and government reports as a reliable autonomous capability to detect, assess and respond to near-field objects in the UAV’s path of travel and the ability to process large volumes of data collected by multiple UAVs in near real-time.

The findings of this study indicate that both of these needs can be better addressed by AI-based information-centric software than legacy data-processing software systems. While AI-based software is not a magic wand that can today fulfill all of the control and data processing needs of unmanned aircraft systems (UAS), it has reached a sufficient level of maturity to warrant immediate exploratory implementation. It is important that the developers of AI-based software become aware of the needs and capabilities of UAS users and manufacturers, and that vice versa, the UAS community become familiar with AI concepts and implementation principles. Since both UAS and AI are relatively young technologies there is a need for the early teaming of members of these communities to guide the research and development efforts and thereby accelerate the effective integration of AI capabilities in UAS software systems.

The report is divided into eight main sections. Section 1 introduces intelligent software as a paradigm shift that is manifesting itself in the transition of data-processing to knowledge management and then states the purpose of this feasibility study. Section 2 briefly summarizes the primary military applications of UAS capabilities and lists the principal technical challenges identified in the literature. In particular, this Section provides an analysis of the Unmanned Systems Roadmap 2007-2032 published in December 2007 by the Office of the Secretary of Defense (OSD 2007). While this publication mentions the need for and promise of AI technology, it generally underestimates in the opinion of the authors current capabilities and the current level of maturity of AI-based software methodologies. Specifically, it overlooks the need for UAS and AI subject matter experts to team together so that the needs of the UAS community can provide appropriate guidance to potentially useful AI research and development efforts.

Section 3 provides an introduction to selected AI concepts and software architecture design and implementation principles. An explanation of the difference between information-centric and data-centric software stresses the role that context plays in the automated interpretation of data. Such interpretation capabilities are a fundamental prerequisite for the timely and effective merging and management of the very voluminous data streams that are typically collected by UAVs on surveillance and reconnaissance missions. Following an explanation of service-oriented architecture (SOA) design concepts and implementation principles, the purpose and functions of SOA components such as services management framework (SMF), enterprise service bus (ESB), and enterprise services are described in conceptual terms. Section 3 concludes by emphasizing the need for integrating AI methodologies with SOA principles to produce software systems that provide flexibility and interoperability in support of reusable, intelligent tools.

Sections 4, 5 and 6 explain the underlying notions and describe the capabilities of: various kinds of intelligent agents; Web services as an implementation of SOA concepts; and, interoperability bridges for facilitating the meaningful transfer of data. This Section builds on the AI technology underpinnings presented in Section 3 and shows how information-centric software principles become an enabler of intelligent tools and services within a SOA-based system environment. Emphasis is placed on the representation of context to enable the automated reasoning capabilities of rule-based agents for planning, re-planning, truth maintenance, and learning functions. However, intelligent agent implementations for UAV systems are likely to be of a hybrid nature involving a variety of AI methodologies including in addition to rule-based agents also neural networks, genetic algorithms, swarm intelligence, and case-based reasoning.

In Section 7 UAV image interpretation serves as the basis of an example application of AI technology to UAV object detection and tracking. A proof-of-concept demonstration system has been developed as part of this feasibility study, utilizing the on-line Adaboost algorithm developed by Grabner and Bischof (2006). By combining the results produced by several Weak Classifiers the Adaboost algorithm is able to dynamically create a single Strong Classifier capable of providing a significantly more reliable object detection result.

Section 8 describes a Phase II follow-on effort that would result in the development of a prototype Unmanned Vehicle Object Recognition and Interpretation System (UVORIS). The proposed capabilities of UVORIS are focused on object detection, tracking and identification, combining several AI methodologies within a SOA system environment. The technical approach includes case-based reasoning, ontology representation, rule-based agents, statistical object recognition, and semantic database search techniques. Section 8 concludes with task statements and estimated costs for a 10-month Phase II effort.

Click here to view entire document. (1.62 MB)