Open Source Intelligence

Access All Intelligence, All Languages, All the Time
Presented by Abe Lederman, President and CTO
Deep Web Technologies, LLC
IOP ¶06 Sheraton Premier, Tysons Corner, Virginia January 16-20


About Deep Web Technologies (DWT) DWT is a New Mexico based company focused on providing state-of-the-art software solutions which Alliance DOE Office of Science National Agricultural Library . 1999 ‡ Major clients include: ± ± ± ± ± DOE Office of Scientific & Technical Information Defense Technical Information Center Science. ‡ Deployed first ³federated search´ portal in the Federal Government. retrieve. aggregate. and analyze content.

accessible through a large variety of means. in myriad formats. located anywhere.Open Source Intelligence The Problem: ‡ Collecting and analyzing enormous quantities of information in any language. with a majority not accessible through the Internet .

Shared Challenge: OSINT and Knowledge Discovery/Diffusion OSINT Challenges Knowledge Discovery/ Diffusion Challenges DWT for the past six years has been the lead technical organization addressing these challenges in collaboration with DOE Office of Scientific & Technical Information .

expertise and ongoing innovations* to address the challenges of OSINT *Developed in partnership with DOE/OSTI .The DWT Proposition To apply DWT¶s technology.

Challenges in Working with Thousands of Data Sources Locate Reliable Sources Categorize Sources by Content Configure Sources for Searching Maintain Sources .

Challenges in Searching Thousands of Sources Automatically Select Sources to Search Perform Many Searches in Parallel Translate. Analyze and Organize Results Relevance Rank Extract Key Information Cluster/ Visualize .

and visualization modules . analyses. grid-computing based federated search engine ‡ Sophisticated Search Conductor ‡ Supports custom connectors ‡ Multi-tier relevance ranking ‡ Framework accepts integration of advanced linguistic.ResearchAssistantTM DWT¶s State-of-the-art Federated Search Engine ‡ Scalable.

Grid Computing: Distributing the Workload .

Search Conductor Select sources to search Perform search Enough good results? YES Deliver results to user NO YES Can I get more results from ³good´ sources? NO .

Multi-tier Relevance Ranking ‡ QuickRankTM ± Ranks results based on occurrence of search terms in title and snippet ‡ MetaRankTM ± Ranks results utilizing custom algorithms applied to metadata ‡ DeepRankTM ± Downloads and indexes full-text documents . Alliance Consortium of 12 Federal Government Agencies Dept of Agriculture Dept of Commerce Dept of Defense Dept of Education Dept of Energy Dept of Health/Human Services Dept of Interior Environmental Protection Agency NASA National Science Foundation US Government Printing Office National Archives & Records Administration Sponsoring Portal (Access to most of Federal Government R&D .

gov Advanced Search Page .Science. Results Page .

A Document .

generating actionable intelligence. This pilot will result in a portal that aggregates content of different types.Next Steps Identify Sponsors and development partners that can collaborate on the development of a pilot that integrates bestof-breed technologies of value to OSINT. .

com/talks/IOP. NM 87544 http://www.deepwebtech.ppt .Contact Us Abe Lederman 122 Longview Drive Los Alamos.

Sign up to vote on this title
UsefulNot useful