Software Talks Are you listening Post Conference edition

Software Talks Are you listening? Post Conference edition (05 Dec 2014) Creative Commons License How to design your mobile apps by Julian Harty is licensed under a Creative Commons Attribution-Share. Alike 3. 0 Unported License. http: //creativecommons. org/licenses/by-sa/3. 0/deed. en_US Julian Harty

Value Why bother?

Are you working too hard, on the wrong things? We struggle to decide what to test, how much testing is enough, etc. 110% effort doesn’t cut it… www. eurostarconferences. com

Most of our work is wasted effort How much of our testing is on target? www. eurostarconferences. com

Most of our work is wasted effort How much of our testing is on target? 80% of reported bugs not addressed Automated Tests www. eurostarconferences. com

Know your users Custom drink feature removed[1] => 1 star feedback ratings Parallel Kingdom[2] Regular users generate 2. 5 x daily revenues Logos © respective owners www. eurostarconferences. com [1] example from App Quality book [2] example from Tale of Two Apps

Understand the effects • Battery drain varied by 3 x for similar hardware specifications • Custom code added for Kindle Fire to reduce brightness – 40% less battery drain • Higher network latencies reduced interactivity by 40% • Users preferred Wi-Fi – 69% for Parallel Kingdom, 58% for Study. Blue • Tablets 2 x usage • Pull-out keyboard also increased usage www. eurostarconferences. com

Analytics can augment our work • Help us to correct and improve what we do • Reduce waste, reduce latency, • Increase value How • Insights into the app’s behaviour in-the-wild • Feedback loops www. eurostarconferences. com

Volume Network profiling Time (of day) Discover Time (of day) Use www. eurostarconferences. com

Volume Network profiling Time (of day) Transform www. eurostarconferences. com

Raw ingredients / context How’s it all work anyway? Why do I care? What’s involved? What’s special about mobile apps?

Some history Web server logs & analysis Web App Logs Web Server Web Logs W 3 C format One ecosystem www. eurostarconferences. com Graphs

What’s different about mobile? Mobile App Logs Common System Log • • Logs isolated on the device Connection not guaranteed Many more sensors Much more variation www. eurostarconferences. com

Data Collector TOPOLOGY Mobile Apps sending Analytics data Database Filter(s) Analytics Web. Server Overview of Mobile Analytics Each step may be delayed www. eurostarconferences. com Business view

Types of Events Mobile app Analytics Library Internet connection Analytics Collector Events 1: 1 App-initiated m: 1 App-initiated Library-initiated E E 1 … E 4 E E Ea Ea L L www. eurostarconferences. com Analytics Database

Analytical Questions: Past Trends, Defect Reports Information Insight What’s Happened? (Reporting) How & why did it happen? (Factor analysis) Software quality models, bottleneck analysis www. eurostarconferences. com

Analytical Questions: Present Engineering Activity, Benchmarking, Testing Information Insight What’s Happened? (Alerts) What’s the best / worst that can happen? (Modeling / Simulation) Specification refinement, asset reallocation www. eurostarconferences. com

Analytical Questions: Future Extrapolation What will Happen? (Forecasting) What’s the best / worst that can happen? (Modeling / Simulation) Failure prediction models www. eurostarconferences. com Information Insight

Analytics for Software Development Trends, Defect Reports Engineering Activity, Benchmarking, Testing Extrapolation Past Insight Future What’s Happened? (Reporting) What’s Happened? (Alerts) What will Happen? (Forecasting) How and why did it happen? Information Present What is the next best action? What’s the best / worst that can happen? (Modeling / Simulation) (Factor analysis) Software quality models, bottleneck analysis (Recommendation) Specification refinement, asset reallocation http: //research. microsoft. com/pubs/136974/foser-2010 -buse. pdf www. eurostarconferences. com Failure prediction models

Implementation Flow Im p De sig v Try us rio a lem ne en ve l te nt s Fie M ve ld ini nt te s sts s a em os ali s ce pt ab Practical ies p Pro m t Feasible ar ibr Ac g pin p at lu va E ib el ns o s rie ra m m Co al rci e ev ati alu www. eurostarconferences. com le? Useful

Gaining confidence Accuracy & precision

Precision & accuracy Precision: repeatability Accuracy: on target https: //en. wikipedia. org/wiki/Accuracy_and_precision www. eurostarconferences. com

Precision & accuracy • Add images here: precision • And here: accuracy www. eurostarconferences. com

Divergent answers increase doubt • A tale of two three mobile analytics libraries (and what happens when bonuses are on the line…) • Where were the testers (part one)? www. eurostarconferences. com

The Dark Side Of Mobile Analytics

DO NO HARM TO A NEIGHBOUR Bad stuff happened; • Location data collected • Excessive traffic Where were the testers (part two)? A Study of Third-Party Tracking by Mobile Apps in the Wild http: //www. vam. ac. uk/users/node/1777 ftp: //ftp. cs. washington. edu/tr/2012/03/UW-CSE-12 -03 -01. PDF www. eurostarconferences. com

Reducing precision to protect privacy 10 km x 10 km squares From: Capturing Mobile Experience in the Wild: A Tale of Two Apps Figure © ACM www. eurostarconferences. com

Beware the automation bias “When presented with an automated solution 40% of pilots reasoned less or none at all” “Automation bias occurs in decision-making because humans have a tendency to disregard or not search for contradictory information in light of a computergenerated solution that is accepted as correct and can be exacerbated in time critical domains. ” Automation Bias in Intelligent Time Critical Decision Support Systems http: //citeseerx. ist. psu. edu/viewdoc/download? doi=10. 1. 1. 91. 2634&rep=rep 1&type=pdf www. eurostarconferences. com

Necessary but not sufficient Listening is a means to an end

Complementary Feedback Polychrome, richer tones while avoiding cacophony • • • System logs Crash logs App Store ratings In-App feedback Speaking to humans. Oh, and software testing www. eurostarconferences. com

Developer Console (Google Play) www. eurostarconferences. com

Instrument the Ecosystem Application Analytics Ecosystem www. eurostarconferences. com Change Me!

Observe the Behaviours Application Ecosystem Keys & Gestures Network Traffic www. eurostarconferences. com

Correlation? Causation? Appropriate? From: User Interaction-based Profiling System for Android Application Tuning Figure © ACM www. eurostarconferences. com

Breaking-up is hard to do From one thing, to another

Divorce can be messy, even for software Changing the code is the easy part… • What about the data? • And the systems & processes that rely on the data? Act in Haste, Repent at Leisure www. eurostarconferences. com

Two ears to listen Are you willing to try?

Willing to try? • Become “one” with the data [Rob Lambert] • Instrumenting their code, need to learn how to understand use the data [Rob Lambert] • Bringing Dev. Ops to Mobile Apps? • Internet-of-Things is coming [Andy Stanford-Clark] • Be aware of what can go wrong [Isabel Evans] www. eurostarconferences. com

Further reading These books available at: [1, 2] http: //wip. org/ [3] http: //www. appqualitybook. com/ [4] http: //www. howtomeasureanything. com/ www. eurostarconferences. com

Academic References Capturing Mobile Experience in the Wild: A Tale of Two Apps http: //doi. acm. org/10. 1145/2535372. 2535391 User Interaction-based Profiling System for Android Application Tuning http: //doi. acm. org/10. 1145/2632048. 2636091 Automation Bias in Intelligent Time Critical Decision Support Systems http: //citeseerx. ist. psu. edu/viewdoc/download? doi=10. 1. 1. 91. 2634&rep=rep 1&type=pdf A Study of Third-Party Tracking by Mobile Apps in the Wild ftp: //ftp. cs. washington. edu/tr/2012/03/UW-CSE-12 -03 -01. PDF All the papers are freely available online, e. g. using Google Scholar www. eurostarconferences. com

Q&A Now? Later: julianharty@gmail. com Creative Commons License How to design your mobile apps by Julian Harty is licensed under a Creative Commons Attribution-Share. Alike 3. 0 Unported License. http: //creativecommons. org/licenses/by-sa/3. 0/deed. en_US