The Durkheim Project Social Media Risk Bayesian

The Durkheim Project: Social Media Risk & Bayesian Counters Hadoop Summit: June 27, 2013 Chris Poulin: PATTERNS AND PREDICTIONS Alex Kozlov: Cloudera Disclaimers: This material is based upon work supported by the Defense Advance Research Project Agency (DARPA), and Space Warfare Systems Center Pacific under Contract N 66001 -11 -4006. Also supported by, the Intelligence Advanced Research Projects Activity (IARPA) via the Department of Interior National Business Center contract number N 10 PC 20221. The opinions, findings and conclusions or recommendations expressed in this material are those of the authors(s) and do not necessarily reflect the views of the Defense Advance Research Program Agency (DARPA) and Space, the Naval Warfare Systems Center Pacific, or the IARPA, DOI/NBC, or the U. S. Government. © 2013 Patterns and Predictions

Speakers Chris v Principal Investigator, DARPA DCAPS Poulin-Dartmouth Suicide Prediction Team v Former Co-Director, Dartmouth Metalearning Working Group (Theoretical Machine Learning) v Artificial Intelligence Instructor, US Naval War College v Principal, Patterns and Predictions (linguistics and prediction of financial events) … and have now read many suicide notes. Alex v Principal Solutions Architect at Cloudera v Ph. D. from Stanford University. v Data mining and statistical analysis at PATTERNS AND PREDICTIONS SGI, Hewlett-Packard

Suicide is a hard societal problem, but why? Stigma: Victims are socially outcast (i. e. disconnected) Negative Topic: Intense negative emotion. And not a 'sexy' research topic by any means. Freedom of Choice: Ultimately you cant stop someone from risky behaviors, or many other activities that risk self harm. And suicide is the ultimate act of personal risk. Logistics: Even if you know what to look for, there are not enough clinicians to help the number of people suffering. Data privacy issues are as intense, or more so then say banking. Prediction: Accuracy (proper identification), false positives (stigmatization), false negatives (malpractice) Deeper issues? : Recent growth in suicide may be related to something more systemically wrong. Suicide the symptom of something else going on. PATTERNS AND PREDICTIONS

Durkheim v The project is named in honor of Emile Durkheim, a founding sociologist whose 1897 publication of Suicide defined early text analysis for suicide risk. v The team is comprised of a multidisciplinary team of artificial intelligence (machine learning and computational linguistics), and medical experts (psychiatrists). v www. durkheimproject. org PATTERNS AND PREDICTIONS

Our Approach v Social Problem: Opt-In is critical o Clear explanations for consent, no tricky EULAs v Technical Problem: How to build a system that collects, stores, analyzes, and allows clinicians to react at Internet scale? Architecture: 1) Opt-In Interface Layer 2) Data Collection Layer 3) Storage Layer 4) Machine Learning, Phase I 5) Machine Learning, Phase II 6) Automated Intervention PATTERNS AND PREDICTIONS

1) Opt-In Interface Layer We cant overemphasize the role of simplified user participation for consent, and privacy control, in our interface/interaction design. PATTERNS AND PREDICTIONS

2) Data Collection Layer The social media component is handled by a content aggregator (Gigya), and populates a Cassandra database. PATTERNS AND PREDICTIONS

Data Collection Layer, Continued The Cassandra instances were built and maintained (by Scale Unlimited) to handle high throughput storage. However, this is not the final destination of the data. PATTERNS AND PREDICTIONS

3) Storage Layer Eventually, the data is moved to the medical center (behind a HIPAA compliant firewall at Dartmouth). Here it persists for ongoing research. PATTERNS AND PREDICTIONS

4) Machine Learning, Phase I In 2011, we initiated a study with the U. S. Department of Veterans Affairs (VA) to study 3 cohorts of 100 subjects each (Non-Psychiatric, and Suicide Positive). v We developed linguisticsdriven prediction models to estimate the risk of suicide. v These models were generated from unstructured clinical notes v From the clinical notes, we generated datasets of single keywords and multi-word phrases v We were able to predict suicide with 65% accuracy on a small dataset. PATTERNS AND PREDICTIONS

5) Machine Learning, Phase II In 2011, we also initiated a study with Cloudera (Alex Kozlov) on a lightweight machine learning framework for detecting real-time risk at scale. v We wanted a clean statistical model for distributed inference (prediction). v We needed a more lightweight framework than Mahout. v We wanted to be able to tradeoff runtime vs. accuracy. v We wanted the prediction library to be eventually open sourced (Apache license) for the community. ‘Alpha’ Build @ http: //durkheimproject. org/bcount/ By Alex Kozlov <alexvk@cloudera. com> PATTERNS AND PREDICTIONS

What is B-counts today? And Why? http: //www. slideshare. net/Hadoop_Summit/bayesiancounters v Distributed aggregation of user events and correlations to fit into RAM of multiple machines v Smart client: Moves substantial amount of logic to clients v Time: An explicit time dimension to support ‘recency analysis’ v Based on HBase v Previous analysis (Poulin) had indicated that words and correlations are a good predictor of target variable v Need a faster processing/response time (response time beats accuracy of the model)

Time to Answer Value vs. time Examples v Advertising: if you don’t figure what the user wants in 5 minutes, you lost him v Intrusion detection: the damage may be significantly bigger after a few minutes after break-in v Mental health risk: you need to screen before negative actions occur http: //cetas. net http: //www. woopra. com http: //www. wibidata. com/

Solution: Time Stamped Hadoop What if we want to access more recent data more often? • Key: subset of variables with their values + timestamp (variable length) • Value: count (8 bytes) index Key Val 1 ue 2 ue 3 ue 4 ue Column families are different HFiles (30 min, 2 hours, 24 hours, 5 days, etc. ) Pr(A|B, last 20 minutes)

A Bayesian Counter, in detail Region (divide between) Counter/Tabl e File Iris [sepal_width=2; class=0] 30 mins 1321038671 Column family Column qualifier Version 1321038998 15 2 hours … Value (data)

Command Line Implementation

Syntax nb iris class=2 sepal_length=5; petal_length=1. 4 300 Target Variable Time (seconds from now) Predictors

Current Classifier Support (alpha release) Πi Pr(F |C) v Naïve Bayes: Pr(C|F 1, F 2, . . . , FN) =1/z Pr(C) v Association rules: Confidence (A -> B): count(A and B)/count(A), Lift (A -> B): count(A and i B)/(count(A) x count(B)) v Nearest Neighbor: P(C) for k nearest neighbors, count(C|X) = ΣXi count(C|Xi), where X 1, X 2, . . . , XN are in the vicinity of X v Clique ranking: I(X; Y)=ΣΣp(x, y)log(p(x, y)/p(x)p(y), Where x in X and y in Y, Using random projection can generalize on two abstract subsets of Z

Performance v retail. dat example – 88 K transactions over 14, 246 items o Mahout FPGrowth – 0. 5 sec per pattern (58, 623 patterns with min support 2) o 10 ms per pattern on a 5 node cluster

6) Intervention Automated systems are coming online for potential patients and families seeking treatment, as well as passive intervention strategies (‘safety plans’). PATTERNS AND PREDICTIONS

What's next? In 2013, we plan a variety of initiatives including the launch of our clinical observation study, deployment of Bayesian Counters on live data, and to seek approval for an automated intervention study. v Launch Data Collection Study (CPHS #23781)… very soon v Deployment of B-Counts on live data for live monitoring v Intervention Research (Clinical Study Approval) PATTERNS AND PREDICTIONS

Conclusion What is Durkheim? And what is the Bayesian Counters library? A near real-time classification library, that, while under development, you’re free to use. Hope that some help is coming to those in need… PATTERNS AND PREDICTIONS

Team Chris Poulin, Director & Principal Investigator Paul Thompson, Study Co-Principal Investigator Thomas W. Mc. Allister, M. D. , Key Personnel Ben Goertzel, Ph. D. , Key Personnel Brian Shiner, MD, Key Personnel Craig J. Bryan, Psy. D, Advisor Linas Vepstas – Lead Machine Learning Programmer Brian Nauheimer – Technical Project Manager Chhean Saur – Lead Web/API Programmer Kevin Watters – Principal Programmer, Middleware Ken Krugler – Lead Distributed Systems Expert Ann Marion – User Experience (UX) Design Jane Nisselson – User Interface (UI) Design Andrew Chen – Social Media Applications Developer Alex Kozlov – Real-time/Distributed Classifier Development Vivek Magotra – Cassandra Database Developer PATTERNS AND PREDICTIONS

THANK YOU Chris Poulin, Managing Partner, Patterns and Predictions chris@patternsandpredictions. net Alex Kozlov, Principal Solutions Architect, Cloudera alexvk@cloudera. com Note: We hope that you have found this talk useful and encouraging. However, if you are having thoughts of harming yourself, please call the Veterans Crisis Line at 1 -800 273 -8255 or 911. © 2013 Patterns and Predictions PATTERNS AND PREDICTIONS