Continuous Integration

• is a software development practice
• members of a team integrate their work frequently, usually each person integrates at least daily - leading to multiple integrations per day
• Each integration is verified by an automated build (including test) to detect integration errors as quickly as possible
• Many teams find that this approach leads to significantly reduced integration problems and allows a team to develop cohesive software more rapidly
• Although Continuous Integration is a practice that requires no particular tooling to deploy, we've found that it is useful to use a Continuous Integration server
• Semaphore website:
  • veryone merging code changes to a central repository multiple times a day. Each merge triggers an automated build and testing sequence for the given project
  • CI acts as a safety net that lets developers prevent many issues before they reach users. As a result, developers ship code with more confidence, but not necessarily faster — the deployment process may still be manual, long, and error-prone
  • we strongly believe that developers must get CI results within 10 minutes, otherwise their productivity shrinks due to lack of focus and frequent context switching
• Atlassian website
  • Ds practicing continuous integration merge their changes back to the main branch as often as possible. Ds changes are validated by creating a build and running automated tests against the build
  • Continuous integration puts a great emphasis on testing automation to check that the application is not broken whenever new commits are integrated into the main branch
• STEPS
  • 0. development of a small feature. Let's assume is small and can be done in a few hours
  • 1. taking a copy of the current integrated source onto my local development machine
    • using a source code management system by checking out a working copy from the mainline
    • source code control systems: keeps all of a project's source code in a repository
    • mainline: current state of the system
    • 'checking out': a controlled copy of the mainline onto the developer machine
    • 'working copy': the copy on the developer's (Most of the time you actually update your working copy to the mainline - in practice it's the same thing.)
  • 2. Now I take my working copy and do whatever I need to do to complete my task
    • This will consist of both altering the production code, and also adding or changing automated tests. Continuous Integration assumes a high degree of tests which are automated into the software: a facility I call self-testing code
  • 3. Once I'm done (and usually at various points when I'm working) I carry out an automated build on my development machine
    • This takes the source code in my working copy, compiles and links it into an executable, and runs the automated tests
    • Only if it all builds and tests without errors is the overall build considered to be good
  • 4.With a good build, I can then think about committing my changes into the repository
  • 5. The twist, of course, is that other people may, and usually have, made changes to the mainline before I get chance to commit
  • 6. So first I update my working copy with their changes and rebuild
  • 6b. If their changes clash with my changes, it will manifest as a failure either in the compilation or in the tests. In this case it's my responsibility to fix this and repeat until I can…
  • 6c. build a working copy that is properly synchronized with the mainline
  • 7. However, my commit doesn't finish my work. At this point we build again, but this time on an integration machine based on the mainline code.
    • Only when this build succeeds can we say that my changes are done (the CI part…)
    • There is always a chance that I missed something on my machine and the repository wasn't properly updated
    • Only when my committed changes build successfully on the integration is my job done
    • This integration build can be executed manually by me, or done automatically by Cruise.
  • NOTE:
    • If a clash occurs between two developers, it is usually caught when the second developer to commit builds their updated working copy (6c)
    • If not the integration build should fail (7)
    • Either way the error is detected rapidly. At this point the most important task is to fix it, and get the build working properly again
    • In a Continuous Integration environment, you should never have a failed integration build stay failed for long
    • A good team should have many correct builds a day
    • Bad builds do occur from time to time, but should be quickly fixed
  • The result:
    • there is a stable piece of software that works properly and contains few bugs
    • Everybody develops off that shared stable base and never gets so far away from that base that it takes very long to integrate back with it
    • Less time is spent trying to find bugs because they show up quickly
• Key practices that make up effective CI
  • Maintain a Single Source Repository
  • Automate the Build
  • Make Your Build Self-Testing
  • Everyone Commits To the Mainline Every Day
  • Every Commit Should Build the Mainline on an Integration Machine
  • Fix Broken Builds Immediately
  • Keep the Build Fast
  • Test in a Clone of the Production Environment
  • Make it Easy for Anyone to Get the Latest Executable
  • Everyone can see what's happening
  • Automate Deployment
• Benefits of Continuous Integration
  • the greatest benefit is reduced risk
  • The trouble with deferred integration
    • it's very hard to predict how long it will take to do
    • and worse it's very hard to see how far you are through the process
    • The result is that you are putting yourself into a complete blind spot right at one of tensest parts of a project - even if you're one of the rare cases where you aren't already late
  • Continuous Integration completely finesses this problem (deferred integration)
    • There's no long integration, you completely eliminate the blind spot
    • At all times you know where you are, what works, what doesn't, the outstanding bugs you have in your system
• Bugs
  • these are the nasty things that destroy confidence and mess up schedules and reputations
  • Bugs in WIP get in your way, making it harder to get the rest of the software working correctly
  • Bugs in deployed software make users angry with you
  • CI doesn't get rid of bugs, but it reduce them does make them dramatically easier to find and remove
  • If you introduce a bug and detect it quickly it's far easier to get rid of
    • Since you've only changed a small bit of the system, you don't have far to look
    • Since that bit of the system is the bit you just worked with, it's fresh in your memory - again making it easier to find the bug
    • You can also use diff debugging - comparing the current version of the system to an earlier one that didn't have the bug
  • Bugs are also cumulative
    • The more bugs you have, the harder it is to remove each one
    • This is partly because you get bug interactions, where failures show as the result of multiple faults - making each fault harder to find
    • It's also psychological - people have less energy to find and get rid of bugs when there are many of them - the Broken Windows syndrome
  • CI tend to have dramatically less bugs, both in production and in process
  • However I should stress that the degree of this benefit is directly tied to how good your test suite is
  • You should find that it's not too difficult to build a test suite that makes a noticeable difference
  • Usually, however, it takes a while before a team really gets to the low level of bugs that they have the potential to reach. Getting there means constantly working on and improving your tests
• CI removes one of the biggest barriers to frequent deployment
  • Frequent deployment is valuable because it allows your users to get new features more rapidly, to give rapid feedback on those features, and generally become more collaborative in the development cycle
  • helps break down the barriers between customers and development - barriers which I believe are the biggest barriers to successful software development
• Introducing Continuous Integration - where do you start?
  • The full set of key practices I outlined above give you the full benefits
  • but you don't need to start with all of them
  • There's no fixed recipe here - much depends on the nature of your setup and team
  • But here are a few things that we've learned to get things going
  • A. get the build automated
  • B. Introduce some automated testing into your build
  • C. Try to speed up the commit build
  • D. If you are starting a new project, begin with CI from the beginning
  • E. Above all get some help
• MAP