Software Merging An Overview Dr Tom Mens Programming

Software Merging An Overview Dr. Tom Mens Programming Technology Lab Vrije Universiteit Brussel Course OOSE. RC EMOOSE 1999 -2000

Problem Statement n Collaborative Software Development u u u n Software merging u u u February, 2000 many different software developers working simultaneously on the same software parallel changes are made to the same code need to combine these changes automated tool support for combining parallel changes detect inconsistencies or unexpected interactions between parallel changes provide support for resolving these inconsistencies Programming Technology Lab, Vrije Universiteit Brussel

Context n n Merge tools are usually part of a configuration management or version management system Definition u “Software configuration management (SCM) is the discipline of managing and controlling change in the evolution of software systems” F n Examples u u u February, 2000 IEEE Standard 1042, 1987 Revision Control System [Tichy 1985] Concurrent Version System [Berliner 1990] Perforce [www. perforce. com] Clear. Case [Leblang 1994] Adele [Estublier&al 1994] Programming Technology Lab, Vrije Universiteit Brussel

Version Terminology Version Revision 1. 0 1. 1 Baseline Parallel changes 1. 2 Merge 1. 3 3. 0 3. 1 Variants 2. 1 February, 2000 2. 2 2. 3 Programming Technology Lab, Vrije Universiteit Brussel

SCM n Traditionally, SCM was seen purely as a management discipline u n Nowadays, it is also treated as a software development support discipline u u February, 2000 [Bershoff&al 80] provides automated help to reduce complexity of making changes to large-scale software systems SCM is necessary in all phases of software life -cycle Programming Technology Lab, Vrije Universiteit Brussel

SCM Concepts n Configuration item u u n a self-contained software artefact whose evolution needs to be tracked and controlled some items can be composite, consisting of other items Version u u u identifies the state of a configuration item at a well-defined point in time each state has a unique version number variants are versions that are intended to coexist F F u a promotion is a version available to other developers F u promotions are stored in a workspace (or dynamic library) a release is a version available to clients or users F February, 2000 e. g. , Mac/Windows/Unix variant of software application e. g. , Light/Standard/Professional edition of software application releases are stored in a repository (or static library) Programming Technology Lab, Vrije Universiteit Brussel

SCM Concepts ctd. n Configuration u n Change request u n formal request for modifying a configuration item Baseline u n a version of a composite configuration item, containing a consistent set of other configuration item versions Formally reviewed and agreed on configuration item that can only be changed through a change request Branch u u concurrent development path requiring independent SCM different branches can be reconciled by merging their versions See [Bruegge&Dutoit 2000]. February, 2000 Programming Technology Lab, Vrije Universiteit Brussel

Exercise 1 SCM systems such as RCS and CVS use file names and their paths to identify configuration items. Explain why this feature prevents the configuration management of composite configuration items, even in the presence of labels. February, 2000 Programming Technology Lab, Vrije Universiteit Brussel

SCM Activities n Configuration item identification u n Status accounting u n each item has unique version number record status of individual components, work products and change requests Build management u enable automatic rebuilding of system when new versions of components are created F n Process management u implement change policy F n February, 2000 minimise amount of recompilation e. g. only syntactically correct code can be part of a version / builds should be made every week / relevant developers should be notified about new versions that have been created Promotion management Programming Technology Lab, Vrije Universiteit Brussel

SCM Activities ctd. n Release management u u creation of releases is decided at management level, based on marketing and quality control advice creation of releases includes: F F F n updating user manual (documentation) ensuring there are no inconsistencies validate completeness and quality Change management u u ensure consistency with project goals during changes different steps 1. request a change 2. assess request against project goals • may include cost analysis and impact analysis 3. accept or reject request 4. plan accepted change, prioritise, and assign to developer 5. audit implemented change (quality control) February, 2000 Programming Technology Lab, Vrije Universiteit Brussel

SCM Activities ctd. n Branch management u merging is needed to coordinate overlapping or interacting parallel changes F u heuristics to minimise merge conflicts F F F n anticipate where overlapping changes can occur merge frequently to identify overlaps early communicate likely conflicts to relevant developers minimalise changes in main branch, and do important changes in separate development branches minimise number of branches Variant management u variants are needed when F F u software operates on different platforms (different OS or hardware) sofware is delivered in variants with different levels of functionality variants can be dealth with by F F February, 2000 detect and resolve conflicts between overlapping changes Different teams for each variant --> reduced complexity / increased redundancy Single project with variant-specific code Programming Technology Lab, Vrije Universiteit Brussel

Roles in SCM n Configuration manager u u n Change control board member u u n u u implements change requests creates promotions resolve merge conflicts Auditor u u u February, 2000 approves or rejects change requests assesses the changes and plans accepted changes Developer u n identifies configuration items defines procedures for creating promotions and releases ensure quality of changes select and evaluate promotions for a release ensure consistency and completeness of a release Programming Technology Lab, Vrije Universiteit Brussel

Storing subsequent versions n Alternatives for storing subsequent versions of a software artefact u u storing all versions integrally using deltas, i. e. , store differences only F forward deltas record original version and apply deltas to produce newer versions • e. g. SCCS [Rochkind 1975] F backward deltas record latest version entirely and apply deltas to produce older versions • e. g. RCS [Tichy 1985] February, 2000 Programming Technology Lab, Vrije Universiteit Brussel

Exercise 2 Most version control systems use backward deltas rather than forward deltas to store subsequent revisions of the same version. Explain why this is the case. February, 2000 Programming Technology Lab, Vrije Universiteit Brussel

Kinds of Merging n n 2 -way vs 3 -way merging reuse versus evolution textual, syntactic or semantic merging state-based vs change-based

a) 2 -way vs 3 -way merging v 1. 0 . . . a. . . v 1. 1 b v 1. 1 a . . . b. . . a. . . 3 -way merge 2 -way v 1. 2 . . . b ? . . . February, 2000 Programming Technology Lab, Vrije Universiteit Brussel

b) reuse vs evolution n merging is necessary u when an object-oriented framework is being customised by a framework user, while it is also evolved by the framework developer F u February, 2000 Cf. reuse contracts When two parallel changes to the same software artifact need to be combined Programming Technology Lab, Vrije Universiteit Brussel

c) textual, syntactic, semantic n textual merging u n syntactic merging u n Use more structured information of software artefacts (e. g. trees or graphs) semantic merging u February, 2000 Considers sofware artefacts as pure text files (or, alternatively, binary files) Use behavioural information about software artefacts Programming Technology Lab, Vrije Universiteit Brussel

text-based merging n Different levels of granularity u Line-based merging takes lines as primitive building blocks F u u Usingle characters as building blocks is too inefficient for primitive use More efficient (two-way) approaches for merging binary files F February, 2000 E. g. Unix diff E. g. bdiff [Tichy 84] and vdelta Programming Technology Lab, Vrije Universiteit Brussel

Exercise 3 CVS uses a simple line-based merge rule to identify merge conflicts: there is a conflict if the same line was changed in both revisions. If no such line exists, no conflict is generated and the merge is performed automatically. a) Explain why this approach fails to detect certain types of conflicts. Provide an illustrative example of both a syntactic and a semantic conflict that goes undetected. b) Vice versa, try to find an example where the approach generates a conflict while there isn’t one. February, 2000 Programming Technology Lab, Vrije Universiteit Brussel

syntactic merging n Based on parse trees u u essentially models “is-part-of” relation between software entities Examples: F [Westfechtel 1991] • domain-independent approach F F [Asklund 1994] Cdiff [Grass 1992] • for parse trees of C++ programs February, 2000 Programming Technology Lab, Vrije Universiteit Brussel

syntactic merging n Based on graphs u More flexible than trees F u also models relations like “invokes”, “calls”, “uses”, “accesses”, “defines”, . . . Examples F F [Rho&al 1998] Reuse contracts • [Steyaert&al 96]: essentially method calls • [Mens 2000]: domain-independent formalism February, 2000 Programming Technology Lab, Vrije Universiteit Brussel

semantic merging n Finding all possible semantic conflicts is an undecidable problem in general u Conservative approaches provide a safe approximation F F u Lightweight approaches only consider part of the semantics F n Can give rise to false positives and false negatives Possible approaches: u using predicates: pre/postconds, invariants, obligations, exceptions F u u [Hoare 69], [Perry 87] using algebraic specifications F February, 2000 No false negatives: all semantic conflicts are detected E. g. [Horwitz&al 89, Binkley&al 95] Larch [Guttag&al 5] . . . Programming Technology Lab, Vrije Universiteit Brussel

d) state-based vs changebased n state-based merging u n only uses information in original version and its revisions change-based merging u u explicitly documents the changes that have been made to the versions extensional change-based versioning annotates the changes inside the version F u intensional change-based versioning describes the changes separately from the versions, in terms of the operations or transformations that have been used. F February, 2000 e. g. [Asklund 1994] E. g. EPOS [Gulla&al 1991] Programming Technology Lab, Vrije Universiteit Brussel

Exercise 4 Explain why intensional change-based merging is more general or more expressive than statebased merging. Also give an example of a conflict that can be detected with change-based merging, but not with state-based merging. February, 2000 Programming Technology Lab, Vrije Universiteit Brussel

Two Definitions of Merging n Two parallel modifications M 1 and M 2 of the same software artifact can be merged if: u u They can be serialised in any order (M 1; M 2 and M 2; M 1), and both serialisations lead to the same result The can be serialised in at least one order (M 1; M 2 or M 2; M 1). M 2 M 1

Exercise 5 a) Give an example of a situation that can be merged by making use of definition 2, but not by means of definition 1. b) Give an example where the merge according to definition 2 leads to a counter-intuitive result. February, 2000 Programming Technology Lab, Vrije Universiteit Brussel

Other Merge Issues n n n Domain-independence Scalability Degree of formality Level of granularity Resolving conflicts Minimising conflicts

1) domain-independence n Most approaches are restricted to a particular programming language u u n Domain-independent approaches: u u February, 2000 Cdiff restricted to C++ Rational Rose Visual Differencing restricted to UML [Westfechtel 1991], using parse trees [Mens 1999], using graphs Programming Technology Lab, Vrije Universiteit Brussel

2) Scalability n Text-based merge tools are not scalable u n changes to multiple lines simultaneously lead to conflicts for each line involved For operation-based merging u Define composite transformations in terms of more primitive ones F u February, 2000 Gives higher-level view of the evolution Ignore some basic conflicts when they appear as part of a composite transformation Programming Technology Lab, Vrije Universiteit Brussel

3) Degree of formality n Ad-hoc u n E. g. Line-based merge tools Lightweight approach u Using conflict tables F u Using graph rewriting F n [Feather 1989], [Steyaert&al 96] [Mens 1999]: confluency & pushout property, parallel & sequential independence Completely formal techniques u [Berzins 1994] F u Denotational semantics and Browerian algebras [Horwitz&al 89, Binkley&al 95] u program slicing February, 2000 dependence graphs and program Programming Technology Lab, Vrije Universiteit Brussel

4) Level of granularity n text-based merge tools u u u February, 2000 line-based block-based character-based Programming Technology Lab, Vrije Universiteit Brussel

5) Resolving Conflicts n Use default conflict resolution strategies u February, 2000 Cf. [Asklund 1994] Programming Technology Lab, Vrije Universiteit Brussel

6) Minimising Conflicts n Small changes can have large impact u n February, 2000 A simple change can give rise to conflicts throughout the entire code Exercise 6: Try to find a number of different ways in which one might consider to reduce the number of detected conflicts to a managable number. Programming Technology Lab, Vrije Universiteit Brussel

Useful Algorithms n Redundancy removal u u n Normalisation u n Canonical form Reconstruction u February, 2000 Reduces number of detected conflicts reduces spaces, increases speed, increases understandability Reconstruct transformation given base version and revised version only Programming Technology Lab, Vrije Universiteit Brussel

Classify existing approaches Considered approach . . . Reuse contracts [Mens 2000] 2 -way / 3 -way text / syntactic / semantic Syntactic: uses typed graphs + Light semantics state / change-based: uses RC operations Domain (in) dependence February, 2000 3 -way independent of considered domain Programming Technology Lab, Vrije Universiteit Brussel

Assignment n Classify a number of approaches according to the given criteria and answer the following questions u u Is the approach 2 -way or 3 -way? Is the approach textual, syntactic or semantic? F u Be as precise as possible: Is it line-based? Which kind of syntactic or semantic software artefacts does it address? Which kind of semantics? (Conservative/light) Is the approach state-based or change-based? F If change-based, is it extensional or intentional? • u Is the approach domain-independent or domain-specific? F u u February, 2000 If domain-specific, can the technique be generalised to more domainindependent artefacts? Wy (not)? Does the approach have a formal foundation? F u If intentional, which operations or transformations are available? Is it scalable to composite transformations? Which? What are the benefits of this? How are conflicts detected? Are there any typical or special features of the approach? Programming Technology Lab, Vrije Universiteit Brussel

Example: reuse contract approach u u 3 -way syntactic approach: specialisation interfaces in [Steyaert&al 96], collaboration diagrams in [Lucas 97], graphs in [Mens 99] Light semantics. . . Change-based merging F F u [Steyaert&al 96] and [Lucas 97] are domain-specific F F u February, 2000 Make use of a conflict table class inheritance hierarchies and collaborating classes, respectively [Mens 99] presents domain-independent formalism based on graph rewriting F u Primitive transformations are Extension, Refinement, Cancellation, Coarsening Composite transformations can be defined Gives a formal characterisation of merge conflicts Special feature: Originally designed for reuse versus evolution conflicts Programming Technology Lab, Vrije Universiteit Brussel

Assignment ctd. § Discuss 3 approaches from the following list: u u u u February, 2000 [Feather 1989] Unix diff & diff 3 utility, Emacs emerge tool, bdiff, vdelta, Sun’s filemerge tool [Adams et al. 1986] [Westfechtel 1991], [Asklund 1994], Cdiff [Grass 1992] Rational Rose Visual Differencing tool SCCS [Rochkind 1975], DSEE [Leblang et al. 1984], RCS [Tichy 1985] Commercial configuration management tools: Clear. Case [Leblang et al. 1988, Leblang 1994], Adele [Estublier et al. 1994] [Horwitz et al. 1989, Binkley et al. 1995], Semantic Diff [Jackson et al. 1994], [Berzins 1994] [Lie et al. 1989], [Lippe et al. 1992] Programming Technology Lab, Vrije Universiteit Brussel

References n n n n n [Adams et al. 1986] E. Adams, W. Gramlich, S. Muchnick, S. Tirfing. Sun. Pro: Engineering a practical program development environment. Proc. Int. Workshop on Advanced Programming Environments. LNCS 244: 86 -96, Springer-Verlag, 1986 [Asklund 1994] U. Asklund. Identifying conflicts during structural merge. Proc. Nordic Workshop on Programming Environment Research ‘ 94, pp. 231 -242, Lund University, 1994 [Berliner 1990] B. Berliner. CVS II: parallelizing software development. Proc. USENIX Conf. , pp. 22 -26, 1990 [Bersoff et al. 1980] E. H. Bersoff, V. D. Henderson, S. G. Siegel. Software configuration management: an investment in product integrity. Prentice Hall, 1980. [Berzins 1994] V. Berzins. Software merge: semantics of combining changes to programs. ACM Transactions on Programming Languages and Systems, 16(6): 1875 -1903, ACM Press, 1994 [Binkley et al. 1995] D. Binkley, S. Horwitz, T. Reps. Program integration for languages with procedure calls. ACM Transactions on Software Engineering and Methodology, 4(1): 3 -35, ACM Press, 1995 [Estublier et al. 1994] J. Estublier, R. Casallas. The Adele configuration manager. In Configuration management: trends in software. John Wiley & Sons, 1994 [Feather 1989] M. Feather. Detecting interference when merging specification evolutions. ? ? ? , pp. 169 -176, ACM Press, 1989 [Grass 1992] J. E. Grass. Cdiff: A syntax directed Diff for C++ programs. Proc. USENIX C++ Conf. , pp. 181 -193, 1992 February, 2000 Programming Technology Lab, Vrije Universiteit Brussel

References ctd. n n n n [Gulla et al. 1991] B. Gulla, E. -A. Karlsson, D. Yeh. Change-oriented version descriptions in EPOS. Software Engineering Journal 6(6): 378 -386, 1991. [Horwitz et al. 1989] S. Horwitz, J. Prins, T. Reps. Integrating non-interfering versions of programs. ACM Transaction on Programming Languages and Systems, 11(3): 345 -387, ACM Press, 1989 [Jackson et al. 1994] D. Jackson, D. A. Ladd. Semantic Diff: A tool for summarizing the effects of modifications. Int. Conf. On Software Maintenance. IEEE Press, 1994 [Leblang et al. 1984] D. Leblang, R. Chase. Computer-aided software engineering in a distributed workstation environment. SIGPLAN/SIGSOFT Software Engineering Symposium on Practical Software Development Environments. ACM SIGPLAN Notices pp. 104 -112, ACM Press, 1984 [Leblang et al. 1988] D. Leblang, R. Chase, H. Spilke. Increasing productivity with a parallel configuration manager. Proc. Int. Workshop on Software Version and Configuration Control, pp. 21 -38, Teubner-Verlag, 1988 [Leblang 1994] D. Leblang. The CM challenge: configuration management that works. In Configuration management: trends in software. John Wiley & Sons, 1994 [Lie et al. 1989] A. Lie, R. Conradi, T. Didriksen, E. -A. Karlsson. Change-oriented versioning in a software engineering database. Proc. 2 nd Int. Workshop on Software Configuration Management, ACM SIGSOFT Software Engineering Notes, 17: 56 -65, ACM Press, October 1989 February, 2000 Programming Technology Lab, Vrije Universiteit Brussel

References ctd. n n n n [Lippe et al. 1992] E. Lippe, N. van Oosterom. Operation-based merging. Proc. 5 th ACM SIGSOFT Symposium on Software Development Environments. ACM SIGSOFT Software Engineering Notes, 17(5): 78 -87, ACM Press, 1992 [Mens 1999] T. Mens. A Formal Foundation for Object-Oriented Software Evolution. Ph. D Dissertation, Vrije Universiteit Brussel, Belgium, September 1999 [Mens 2000] T. Mens. Conditional graph rewriting as a domain-independent formalism for software evolution, Proc. Int. Agtive ’ 99 Conference, LNCS, Springer-Verlag, 2000 [Rho et al. 1998] J. Rho, C. Wu. An efficient version model of software diagrams. Proc. 5 th Asia. Pacific Conf. Software Engineering, pp. 236 -243, 1998 [Rochkind 1975] M. Rochkind. The source code control system. IEEE Transactions on Software Engineering, 1(4): 364 -370, IEEE Press, December 1975 [Tichy 1985] W. Tichy. RCS: a system for version control. Software Practice and Experience, 15(7): 637 -654, 1985 [Westfechtel 1991] B. Westfechtel. Structure-oriented merging of revisions of software documents. Proc. 3 rd Int. Workshop on Software Configuration Management, pp. 68 -79, ACM Press, 1991 February, 2000 Programming Technology Lab, Vrije Universiteit Brussel