IBM Research Noisy Text Correction – an exercise in futility? Sreeram Balakrishnan IBM India Research Lab
Noisy Text Analytics Workshop IIIT Jan 8 th 2007 Aggregate versus Instance analysis § Can divide applications for noisy text into two broad categories 1. Applications that look at individual text instances Eg Search, transcription (OCR, speech 2 text) 2. Applications that look at aggregate features of the text Eg Document classification, Aggregate text analytics § Aggregate analysis is more robust to noise since errors can be averaged out. Ø Text correction techniques can help improve accuracy of aggregate statistics § Applications that require accurate correction of each text instance may be an exercise in futiliy (at least in the short term) Ø 2 Eg the example of SMSs that require knowledge of whole context of conversation to manually correct
Noisy Text Analytics Workshop IIIT Jan 8 th 2007 Example – Customer Contact Records • IBM PC help centers received over 500, 000 calls per year • Agents produce summary transcripts for each call Date: 19990425, ID: 13163548 PRELA 04/25/1999 20: 46 - Call 13171316 POWER Date: 19990426, ID: started by John Velocci (MOB_NORTH). Date: 19990604, Q: wants to know ID: he's Callby Barryagaint (IREL_MOB 3). Q: Tp attached to dock with if 13376646 MONIT O'Kelly the Mac. Donald protected b 04/26/1999 18: 50 -started by Scott CIH virus. a: has ibm anti 06/04/1999 22: 16 - Call virus installed. Date: 19990605, (MOB_NORTH). ID: 13376581 MONIT and monitor A: Undocked Tp. . . booted. . . screen external monitor. . . black border around LCD 06/03/1999 full Only getsgoto the web site Robert Dennis patches. told monitor oow A: q: CX'S DOG ATE- HER POWER SUPPLY Was reinstallinghim toiswhen cus told him. Date: 10: 39 when attached byfor upgrade (MOB_NORTH). Machine fax warranty to border Call started to 8629697 19991001, out dock and monitor Q: machine lcd panel will cut. ID: on cust with placed in Mobiles call back queue by Barry disconnected S: Training 06/04/1999 23: 11 - Call the machine sitting normally cust states that pop. s: self st: Date: 19991051, ID: or 8630655 Andrewby Johnher to keep machine a: looked (IREL_MOB 3). side of adapter 02 k 6496 F 10 key is what is(TAG 4). to parts the pn for ac machine at near the closed Atias needed Q: Customer presure on the under 06/04/1999 18: 14 - Call and transfered Velocci O'Kelly upclosed 04/25/1999 20: 51 - taken by running. . advised Call repair cust agreed, Mac. Donald calling back, customer still TP 8: memory onseeking (MOB_NORTH). about getting blackby Scott. LCD and monitor. 10: 46: 02 * MSG FROM 04/26/1999 18: 55 - billableclosed borderupgrade. R 3 service A: I explained to the inquiry EZSRV : this will happen when using a simultanious display. S: FROM EZSRV customer that Easy. Serve R 3 pickup request received for 10: 46: 02 * MSG SOP 06/04/1999 : 19: 04 (MOB_NORTH). Atias (TAG 4). advised customer that data may be lost when sending Machine Type: 2640 about TPxx - Call closed by Andrew Serial# 78 GM 283 large number of complaint in to ez serve. . advised. Information for Purchased Equipment CALL TYPE: Technical customer to back up all TP 8: memory but also machine may customer would like to upgrade personal data. . . if possibleneeds information. customer claim logs be reloaded MICH-2 CODENAME: as part of pd/repair process please have all software, product licenses and COA's available when MACHINE TYPE: machine is returned also please write the case number on the outside of the box 9546 before sending in gave customer information about memory products. OMPONENT TYPE: to ez serve repair 06/03/1999 10: 49 - Call closed by Robert Dennis Monitor/Display (MOB_NORTH). 06/04/1999 22: 52 - Case Number: 13366697 continued by Margaret Butler : : customer reports paint is peeling around palm rest diverse contact media • Summary records contains details of why customers are unhappy • Aggregate analysis of the key phrases reveals that paint peeling at palm rest is common complaint of many TPxx users 3
Noisy Text Analytics Workshop IIIT Jan 8 th 2007 Value from wider sources of data Customer Care How can customer service Customer survey: I was unhappy with service be improved? because the agent did not answer my question Market Intelligence How can campaigns for Customer contact: I would like information on upsell be better targeted? mutual funds Quality Insights Why are warranty claims high for product XX Claim form: Hinge cracked on product XX after 3 months Epinions. com: Don’t buy product XX, the hinge cracks within 1 year Research & Discovery What diseases are associated with gene-xyx Medline documents: Gene-xyx expressed in mouse with zzz 4
Noisy Text Analytics Workshop IIIT Jan 8 th 2007 Some example of SMS data § § § § § 5 Please send me about yyy card What is the no. that i may have to dial for kaun banega lakhpati ? Tell me about new plans plese thanks Mera custmbar care kayu band kar diya gaya hai kirpa karke aap mera custmbar chalu karen kayu ki mera ko newplan ki jankare chahi Sir mere custmer care nall gall nahi ho rahi. . Please activate my wap over gprs Gup shup pack not activating. unable to connect 656. Pl. confirm the receipt of payment of Rs. 500 paid on 19. 05. 06 vide receipt 0244213 at Karanagar. Thanks Request har never made by me for ISD. I dont need ISD 24 hrs over what is the reply I am post paid customor & i have quary about my bill but custamer ex are not there. What can i do now 3 din ho gaye aap ko aap ke 24 hour kab pure honge Xxx ki service ko kya ho gaya hai. custmer ko satisfied hi nahi karte. Tell me where i can contact othewise i would take another connection. The service of xxx is extremly bad and some of the senior employee are irresponsible regarding their work. e. g. (XYZ) Xxx service veri poor. No care for customer is what Xxx focus on. I've to leave xxx as it is not solving my problem. Gudbye Keep NOT care customers I am very distrebed to xxx massangar I riqvest 3 rd time complained Bhaji plz custmar care service chalu kar do nahi ta mai no. Band kar devaga. Menu bahut mushkil aa rahi plz. Kal spice da no chalega
Noisy Text Analytics Workshop IIIT Jan 8 th 2007 Some examples of call centre notes § § § § 6 no. fwd to unbarr pls actv. AR on cst reqt. . . 10: 19 am. cust wants to actv roam as he don't understand the ivr roming actv on cust req , , , , , charges told, , reena no. unbarred as pymt reflected on cust req xxx roaming deactv on cust req the cust secratory called up and he inf tht he was not able to access GPRS , he was not able to confirm whether its masala or MO, and he told that he will call back with other details later and disconn teh call No waiver given to him at any cost. . . . promotional mssg restricted as on cst req. . . 11: 14 am Customer was charged SMS for Rs. 3074. But customer didnt give request for deactivation of 10000 sms pack. Since om dwn, not able to chk active or not. But its shows active in new crm window. resume no. as pymt is reflected. . . . . 9. 20 am ar deactivated on cust request case escallated : HEALTH ALERTS to be deactivated. . . 11: 15 am
Noisy Text Analytics Workshop IIIT Jan 8 th 2007 Enriching structured BI with unstructured data Unstructured Information Management Architecture Unstructured sources UIMA processing Link/Cleanse/Transform Unstructured Enriched Data Warehouse Reporting OLAP tools Modeling Mining Structured sources § § 7 Augment classic structured data warehouses with information extracted from unstructured sources Domain specialized annotators embedded in UIMA (open source) extract structured attributes from unstructured sources
Analysis of Agent Performance (AT 002) Scenario: IBM BPO business with agents handling car rentals wants higher check-out rate Solution: Extract and correlate key phrases from call transcripts with outcomes Value Selling Phrases Mention of Good Vehicle Mention of Good Rate
Analysis of Agent Performance Higher use of value selling phrases mentioning good rate for checked out cars versus no show Value selling phrases Checked out cars 47% No shows 25% Cancelled Pick up information
Скачать презентацию IBM Research Noisy Text Correction an exercise
ca2a7c8a2128db6e1d206e20198bda26.ppt