Buildkite

Buildkite Pty Ltd was founded in September 2013 as Buildbox, with the product out of beta in November.^[4] Tim Lucas came on board as a co-founder.^[5] The original name of the company, Buildbox, had to be changed due to another company registering the trademark first, reportedly by only two weeks.^[6]

In 2022, they announced a new tool, Test Analytics.^[7][8] In 2023, the company makes their first acquisition, PackageCloud, from a private equity firm. PackageCloud allows companies to store and distribute software to clients.

In 2020, Buildkite raised $28M AUD, led by OpenView, with General Catalyst. The company was valued at 200M (AUD) in that deal.^[3][2] In November 2022, the company raised a Series B round of $21M ($32M AUD), with investors including One Ventures, Dominic Pym (angel) and AirTree.^[9][3][10] Total funding investment at the time of writing is $39MUSD.

Buildkite Pipelines, Buildkite's primary product, was built with a hybrid model where build agents are self-hosted, yet supported by a managed, cloud-powered interface. In the hybrid model, the customer provides their own infrastructure for running agents, including their secrets, while Pipelines has a web application to handle user authentication, build orchestration, store logs and metrics.^{[11][12][13][2][14]}

Features of Buildkite Pipelines include parallelized builds,^[15][11] self-hosted agent infrastructure, build visualization, and integrations with services like GitHub,^[16] Bitbucket, GitLab, and Slack.^[17][18][19] Pipelines permits up to 10,000 agents per account for customers.^[13]

The platform allows the creation of pipelines, which are the definitions (in YAML format) for entire automated build processes. A pipeline consists of individual steps, where each step defines an executable job that runs on one or more agents. Agents are the computers, virtual machines, or containerised infrastructure that provide the capabilities to execute these jobs.

An example of a pipeline YAML file, consisting of three steps:^[20]

steps:
  - label: ":hammer: Tests"
    key: "tests"
    command: scripts/tests.sh
    env:
      BUILDKITE_DOCKER_COMPOSE_CONTAINER: app

  - label: ":package: Package"
    depends_on: "tests"
    command: scripts/build-binaries.sh
    artifact_paths: "pkg/*"
    env:
      BUILDKITE_DOCKER_COMPOSE_CONTAINER: app

  - label: ":debian: Publish"
    depends_on: "tests"
    command: scripts/build-debian-packages.sh
    artifact_paths: "deb/**/*"
    branches: "main"
    agents:
      queue: "deploy"

Buildkite Test Analytics provides development teams with data about their test suites through metrics and trend analysis, and integrates with Buildkite Pipelines. Using the input data, Test Analytics identifies flaky tests, visualizes historical test performance, and correlates test runs with builds and deployments.

Test Analytics requires a test collector with a test suite that runs as a part of the pipelines definition. There is a Test Collector Buildkite plugin for JSON and Junit files, and some frameworks have specific test collectors, e.g. specific JavaScript frameworks.^[7][21]

The test collector gathers data about the test suite's execution to report back to Test Analytics for visualization and interpretation through the web application. Data includes the performance and duration of tests, as well as the reliability (or flakiness) of tests.^[7][22][23]

At the time of writing, Packages is in beta/early access. It's marketed as an artifact and package management solution that "supports all major package formats", splits storage from orchestration for security, includes support of the Buildkite command line interface (CLI), among other features.^[24][25][26]

No public release date has been shared.

Buildkite's products consist of the following software components.

The Buildkite web application is the software component and platform that provides the user interface to integrate Pipelines, Test Analytics and Packages. The web application provides the following features:

• Pipeline orchestration
• Build/queue metrics
• Agent utilization
• Test suites
• Artifact/package management (beta)

The agent is a small software component that runs natively on different operating system environments and provides the agent capabilities to run a pipeline's jobs from Buildkite Pipelines. An agent also refers to the computer, virtual machine or containerised infrastructure running the Buildkite agent software component that provides the capabilities to run jobs from a pipeline.

The test collectors are other small software components that integrate with a pipeline's jobs running test suites. A test collector analyzes the performance and reliability of tests run by a test suite and reports the data analytics back to Test Analytics, assuming the customer has that product enabled.