Recovering Temporally Rewiring Networks A Model-based Approach Fan

Recovering Temporally Rewiring Networks: A Model-based Approach Fan Guo, Steve Hanneke, Wenjie Fu, Eric P. Xing School of Computer Science, Carnegie Mellon University 3/16/2018 ICML 2007 Presentation 1

Social Networks Physicist Collaborations High School Dating The Internet 3/16/2018 All the images are from http: //www-personal. umich. edu/~mejn/networks/. That page includes original citations.

Biological Networks Model for the Yeast cell cycle transcriptional regulatory network Fig. 4 from (T. I. Lee et al. , Science 298, 799 -804, 25 Oct 2002) Protein-Protein Interaction Network in S. cerevisiae Fig. 1 from (H. Jeong et al. , Nature 411, 41 -42, 3 May 2001) 3/16/2018 ICML 2007 Presentation 3

When interactions are hidden… Infer the hidden network topology from node attribute observations. Methods: Optimizing a score function; Information-theoretic approaches; Model-based approach … 3/16/2018 Most of them pool the data together to infer a static network topology. ICML 2007 Presentation 4

And changing over time Network topologies and functions are not static: Social networks can grow as we know more friends Biological networks rewire under different conditions Fig. 1 b from Genomic analysis of regulatory network dynamics reveals large topological changes N. M. Luscombe, et al. Nature 431, 308 -312, 16 September 2004 3/16/2018 ICML 2007 Presentation 5

Overview 3/16/2018 Network topologies and functions are not always static. We propose probabilistic models and algorithms for recovering latent network topologies that are changing over time from node attribute observations. ICML 2007 Presentation 6

Rewiring Networks of Genes 3/16/2018 Networks rewire over discrete timesteps Part of the image is modified from Fig. 3 b (E. Segal et al. , Nature Genetics 34, 166 -176, June 2003).

A Graphical Model Counterpart Transition Model Emission Model 3/16/2018 ICML 2007 Presentation 8

Technical Challenges 3/16/2018 Latent network structures are of higher dimensions than observed node attributes How to place constraints on the latent space? Limited evidence per timestep How to share the information across time? ICML 2007 Presentation 9

Energy Based Conditional Probablities 3/16/2018 Energy-based conditional probability model (recall Markov random fields…) Energy-based model is easier to analysis, but even the design of approximate inference algorithm can be hard. ICML 2007 Presentation 10

Transition Model Based on our previous work on discrete temporal network models in the ICML’ 06 SNA-Workshop. Model network rewiring as a Markov process. An expressive framework using energy-based local probabilities (based on ERGM): Features of choice: (Density) 3/16/2018 ICML 2007 Presentation (Edge Stability) (Transitivity) 11

Emission Model in General Given the network topology, how to generate the binary node attributes? Another energy-based conditional model: 3/16/2018 All features are pairwise which induces an undirected graph corresponding to the time-specific network topology; Additional information shared over time is represented by a matrix of parameters Λ; The design of feature function Φ is application-specific. ICML 2007 Presentation 12

Design of Features for Gene Expression The feature function 3/16/2018 If no edge between i and j, Φ equals 0; Otherwise the sign of Φ depends on Λij and the empirical correlation of xi, xj at time t. ICML 2007 Presentation 13

Graphical Structure Revisit Hidden rewiring networks Initial network to define the prior on A 1 Time-invariant parameters dictating the direction of pairwise correlation in the example 3/16/2018 ICML 2007 Presentation 14

Inference A natural approach to infer the hidden networks A 1: T is Gibbs sampling: To evaluate the log-odds Conditional probabilities in a Markov blanket Tractable transition model; the partition function is the product of per edge terms Computation is straightforward Given the graphical structure, run variable elimination algorithms, works well for small graphs 3/16/2018 ICML 2007 Presentation 15

Parameter Estimation Trade-off between the transition model and emission model: 3/16/2018 Larger θ : better fit of the rewiring processes; Larger η : better fit of the observations. Grid search is very helpful, although Monte Carlo EM can be implemented. ICML 2007 Presentation 16

Results from Simulation Data generated from the proposed model. Starting from a network (A 0) of 10 nodes and 14 edges. The length of the time series T = 50. Compare three approaches using F 1 score: 3/16/2018 avg: averaged network from “ground truth” (approx. upper bounds the performance of any static network inference algorithm) ht. ERG: infer timestep-specific networks s. ERG: the static counterpart of the proposed algorithm Study the “edge-switching events” ICML 2007 Presentation 17

Varying Parameter Values 3/16/2018 F 1 scores on different parameter settings (varying ICML 2007 Presentation ) 18

Varying the Amount of Data 3/16/2018 F 1 scores on different number of examples ICML 2007 Presentation 19

Capturing Edge Switching Summary on capturing edge switching in networks 3/16/2018 Three cases studied: offset, false positive, missing (false negative) mean and rms of offset timesteps ICML 2007 Presentation 20

Results on Drosophila Data The proposed model was applied to infer the muscle development subnetwork (Zhao et al. , 2006) on Drosophila lifecycle gene expression data (Arbeitman et al. , 2002). 11 genes, 66 timesteps over 4 development stages Further biological experiments are necessary for verification. Network in (Zhao et al. 2006) 3/16/2018 ICML 2007 Presentation Embryonic Larval Pupal & Adult 21

Summary A new class of probabilistic models to address the problem of recoving hidden, time-dependent network topologies and an example in a biological context. An example of employing energy-based model to define meaningful features and simplify parameterization. Future work 3/16/2018 Larger-scale network analysis (100+? ) Developing emission models for richer context ICML 2007 Presentation 22

Acknowledgement Yanxin Shi CMU Wentao Zhao Texas A&M University Hetunandan Kamisetty CMU 3/16/2018 ICML 2007 Presentation 23

Thank You! 3/16/2018 ICML 2007 Presentation 24