Practical Approaches to Sharing Information at Raytheon Taxonomies

Practical Approaches to Sharing Information at Raytheon Taxonomies, Metadata and Beyond Presented by Christine JM. Connors Christine_Connors@raytheon. com KMPro / KMForum, Bentley College, Waltham, MA June 30, 2004

Data Discovery – What we Learned • Level of information management varies • 85% of our information is unstructured • Over 90% of information is not tagged • High proportion of tagged documents result of templates, and therefore relay bad data • Ethan Frome – over 200 documents • Automatically extracted data imprecise • “The flight to Dallas is cancelled” • Greater than 13% of information is exactly duplicated • “Near” duplication harder to determine but potentially more costly • Worst duplication in File Servers / Shared Drives • Difficult to determine true age of document due to web scripting, date of publication to public drive • Over 23% of sample data not modified in previous 5 years • Over 68% of sample data not modified in previous 2 years 2

Intranet Search and Browse Survey June 2003 • 16 multiple choice questions, 1 optional free-text comment field • 516 surveys started (clicked into) • 199 responses over 3 weeks • 39% completion rate • 101 comments on “How can we improve the intranet search and • browse capabilities? ” 51% comment rate by survey participants 3

How can we improve the intranet search and browse capabilities? 101 user comments frequently included: • Qualify searches by function, organization, and business • Qualify searches by date • Qualify searches by document type (especially web pages) • Provide sorting of results by date, document type • Provide category search • Do not change URLs of pages (users bookmarked) • Reduce number of search results • Google (mentioned 32 times in comments) 4

Results Summary Search/browse • • • About ¼ (26%) of respondents find the current capabilities “Good” or “Excellent” About ¼ (24%) of respondents consistently locate helpful information About ¼ (22%) of respondents indicate they are consistently successful using keyword searching About ¼ (26%) of respondents find it consistently easy to browse The advanced search page is used sparingly Categories • 62% of respondents would find categories consistently useful Bookmarks • • • 36% of respondents consistently return to previously found information 71% of respondents use bookmarks “Most of the time” or “Always” 65% of respondents consistently organize their bookmarks 5

OUCH! • “The search engine is poor to inadequate. I needed to find an appropriations data sheet and was returned 366 entries, none which had anything to do with appropriations. I spend far too much time looking through the search results for this engine to be effective. If I could find this document on the INTERNET I would do so, but this is an internal Raytheon document that is successfully hidden somewhere in the archives with the Ark of the Covenant. ” • Unidentified search and browse survey participant, June, 2003 • “Who gets more hits: www. amazon. com or www. thequaintbookstoredownthestreet. com? Listen up people: Our intranet is a wasteland of information. We need to unify - we need to standardize. Information is power - but only if it is on my desktop, not hidden away in some server waiting for a lucky adventurer to uncover it like some lost continent. ” • Another unidentified search and browse survey participant, June, 2003 6

Usability Testing • Bentley College’s Design and Usability Testing Center • 4 Focus Groups of 8 -10 people each • They told us: • Want to filter searches • Didn’t want long list of items to select from • Liked “Suggested…” boxes • Didn’t understand the taxonomy when presented like Yahoo! • Liked taxonomy as file folder metaphor • Liked thesaurus • Were confused by relational taxonomies • Liked “Categories” as the tab label, over Topics, Taxonomy, Thesaurus, Subjects or Browse. 7

Taxonomies – Who? § The Dream Team § Information Scientists § Cognitive Scientists § Linguists § Programmers § Database Experts § Network Specialists § Verity Administrators § Human Computer Interaction / Usability Experts § Subject Matter Experts § Organizational Change Management § What we got § Information Scientists (in-house) § Cognitive Scientist / Linguist (from Verity) § Programmers, Database Experts, Network Specialists, and Verity Administrators (both in-house and from Verity) § HCI / UI Experts (in-house, Verity and Bentley College) § Subject Matter Experts (in-house) 8

Taxonomies – What for? • • • Browse & Navigation Relational Taxonomies Refine Search Parametric Search Federated Search • • • Dynamic taxonomies Profiling Compliance Engine • • • Classification & Categorization Provide controlled vocabularies to use with Metadata Schema(s) Easy selection to minimize angst over having to fill out file properties 9

Taxonomies 2003 § Deployed 5 taxonomies § Defense Technologies (based on DTIC) § Purchased DTIC taxonomy § Revised to fit Raytheon’s data § Removed several categories including agriculture that are not needed § Raytheon Products § Revised our products listing into a hierarchical approach § Enlisted Raytheon Communicators as Subject Matter Experts § IPDS § Built using data from the IPDS web site § Enlisted IPDS experts as Subject Matter Experts § Engineering § Implemented taxonomy built by Raytheon’s Engineering Technology § Network – Needs revising and enhancing Information Technology § Purchased from Verity § Revised to fit Raytheon’s data § Enlisted members of Corporate IT as Subject Matter Experts 10

Taxonomies 2004 § Human Resources taxonomy § Deployed March 2004 § Purchased from Verity § Cross-functional team of HR representatives reviewed as Subject Matter Experts § Six Sigma § Will be deployed June 2004 § Built based on Raytheon Six Sigma data § Legal Taxonomy § Will be deployed June 2004 § Purchased from Verity § Able to create additional taxonomies for Ethics, Environmental Health & § Safety, and Export/Import Compliance from the purchase of this ONE taxonomy Will be restructuring our top level categories : § Business Units – domestic & international § Functions § Processes § Products § Relationships 11

Taxonomies – How? • • • Card sorting • EZ Sort • 3 x 5 cards Review search engine logs • Internal logs • Webtrends Review organic systems • Web and file share navigation Review existing taxonomies/thesauri Concept Mapping • Linguistic algorithm Intelligent Classifier – lots of query building behind each node Multi. Tes Mind Manager Text. Pad BUY! 12

Build vs. Buy • • • Build to suit users • Reflect corporate vernacular • Internal acronyms • Corporate culture • How is the business structured and portrayed? Can take a long time • Time estimates depend on type/use of taxonomy/tools available • Simple = 5 minutes to build term, 5 minutes to build category and map the topic to the taxonomy • Complex = 75 minutes to build term • PLUS Quality Assurance testing! Buy • Industry standard • Rapid implementation Need customization Both decisions require maintenance 13

Benefits - Increased Productivity Delphi Group 2003 – as reported by Gartner • Business professionals spend more than 2 hours per day searching for information • Half of that time – 1 hour per day is wasted by failure to find what they seek • The single factor most attributed to the large amount of time wasted was • data changes (location 35%) and • bad tools (ineffective search and lack of labeling 28%) • If we conservatively assume only 1 hour per year would be saved per general employee and 1 hour per month per engineer, then: 14

Benefits – Reduce Storage Costs TB TB TB Relative of starting point, growth curves represent storage acquisition cost increases over time. 15

Is it working? • “New” search launched September 29 • Latest survey results show improvement • Neutral rating upgraded to Good • Metrics show increased usage of search • 17% increase in unique users per day • 25% increase in searches per day • Metrics show increasing use of categories in search • Since launch, the categories have been used 50, 000 times • ITLT approved project funding • Knowledge Representation team recipients of 2003 IT Excellence in Collaboration and Knowledge Management Award 16

17 17

18 18

19 19

20 20

21 21

22 22