BitBucket Pipeline

A worked example of a Bitbucket Pipelines build that packages a DataStar deployment and pushes it to Octopus. In this setup:

• The deployment file is attached to a Jira story from the DataStar client (Save to Jira / Export to Jira).
• Each story is worked in a feature branch named after the story key (feature/DAT-243432).
• The pipeline downloads the deployment file, builds a package, pushes it to Octopus, and creates a release.
• A merge to master re-runs the pipeline against the last merged feature branch.

This is one shape the pipeline can take. Equivalent flows work with release branches, approved merge requests, or a shared release folder.

Step 1. Download the deployment file

The first job pulls the deployment file from the Jira attachment. The branch name (feature/DAT-...) gives the story key.

- step:
    name: Download Deployment File
    image: python:3.9.10
    script:
      - pip install requests
      - export STORY=$(echo "$BITBUCKET_BRANCH" | sed 's/.*feature\///')
      - python download-deployment.py -d "$BITBUCKET_CLONE_DIR/output" -o "deployment.xml" -b "https://mydomain.atlassian.net" -u "$JIRA_USERNAME" -p "$JIRA_PASSWORD" -s "$STORY"
    artifacts:
      - output/*.xml

The helper script uses Jira's REST API to read the issue, find the deployment.xml attachment, and download it. The full script is in bitbucket/download-deployment.py.

Step 2. Create the package

DataStar.Tools installs as a global dotnet tool. Its build command reads the deployment file, takes each item out of the repository at the revision it is pinned at, inflates any data document or changeset to SQL, and packs the result as a NuGet package.

- step:
    name: Create Deployment Package
    clone:
      depth: full
    image: mcr.microsoft.com/dotnet/sdk:10.0
    caches:
      - dotnetcore
    script:
      - export PATH="$PATH:/root/.dotnet/tools"
      - dotnet tool install --global DataStar.Tools --version 3.1.0
      - export STORY=$(echo "$BITBUCKET_BRANCH" | sed 's/.*feature\///')
      - >-
        datastar --command-name build
        --log-level debug
        --git-directory "$BITBUCKET_CLONE_DIR"
        --build-file "output/deployment.xml"
        --database Oracle
        --work-item "$STORY"
        --output-directory "$BITBUCKET_CLONE_DIR/build"
        --package-directory "$BITBUCKET_CLONE_DIR/artifacts"
        --package-file 'ORA-${TaskId}.${Version}.nupkg'
        --package-id "ORA-$STORY"
        --package-version "$BITBUCKET_BUILD_NUMBER"
        --package-author "Absolute Technology Ltd"
    artifacts:
      - artifacts/*.nupkg

• clone: depth: full matters: each item is read at the commit it is pinned at, so the history has to be there. Bitbucket's default clone stops after 50 commits.
• The output directory must be empty, so it is not output, where step 1 left the deployment file.
• --database is the vendor the data documents inflate for: Oracle or Microsoft.
• ${TaskId} and ${Version} in the package file name are replaced with the work item and the package version, so the package is written as artifacts/ORA-DAT-243432.42.0.0.nupkg for build 42.

The package holds a metadata.json that build writes, with the work-item reference and version, so downstream Octopus steps can read them via the DataStar release step template's variables-file feature.

Step 3. Push to Octopus

- step:
    name: Deploy to Octopus
    image: octopusdeploy/octo:6.17.3-alpine
    script:
      - octo push --package ./artifacts/*.nupkg --server $OCTOPUS_SERVER --apiKey $OCTOPUS_APIKEY

Step 4. Create the Octopus release

Octopus dynamic package selection means the release step must pin versions for every package in the project:

• The DataStar.Tools package (the deployment CLI).
• The templates package (needed for reversal).
• The scripts package pushed in step 3.

- step:
    name: Create Octopus Release
    image: python:3.9.10
    script:
      - pip install requests
      - export VERSION=$(echo "$BITBUCKET_BRANCH" | sed 's/.*feature\/DAT-//')
      - python octopus-release.py -b "$OCTOPUS_SERVER" -a "$OCTOPUS_APIKEY" -w "$VERSION" -v "$BITBUCKET_BUILD_NUMBER" -p "Oracle Release" -s "DataStar"

The helper script walks the Octopus REST API to resolve the space, project, and channel; then picks the latest published version for each package, except for the dynamically-named one, which uses the value passed in. The full script is in bitbucket/octopus-release.py.

Merge to main

Running the same pipeline against master needs a way to recover the story key, since the branch name no longer encodes it. The approach below reads the last merged feature/... branch via git for-each-ref and hands the key to the next step in a file, since Bitbucket steps share files but not variables. build writes the package's metadata.json itself, so nothing else needs passing on.

The master-branch pipeline fragment:

branches:
  master:
    - step:
        name: Master Branch Deployment File
        clone:
          depth: full
        image: python:3.9.10
        script:
          - pip install requests
          - export STORY=$(echo | git for-each-ref --count=1 --format='%(refname)' --sort=-committerdate refs/remotes/origin/feature --merged | sed 's/.*feature\///')
          - echo "WorkItem=$STORY" && echo "Version=$BITBUCKET_BUILD_NUMBER"
          - mkdir -p "$BITBUCKET_CLONE_DIR/output"
          - echo "$STORY" > "$BITBUCKET_CLONE_DIR/output/story.txt"
          - python download-deployment.py -d "$BITBUCKET_CLONE_DIR/output" -o "deployment.xml" -b "https://your-company.atlassian.net" -u "$JIRA_USERNAME" -p "$JIRA_PASSWORD" -s "$STORY"
        artifacts:
          - output/*.xml
          - output/story.txt
    - step:
        name: Create Deployment Package
        clone:
          depth: full
        image: mcr.microsoft.com/dotnet/sdk:10.0
        caches:
          - dotnetcore
        script:
          - export PATH="$PATH:/root/.dotnet/tools"
          - dotnet tool install --global DataStar.Tools --version 3.1.0
          - export STORY=$(cat output/story.txt)
          - >-
            datastar --command-name build
            --log-level debug
            --git-directory "$BITBUCKET_CLONE_DIR"
            --build-file "output/deployment.xml"
            --database Oracle
            --work-item "$STORY"
            --output-directory "$BITBUCKET_CLONE_DIR/build"
            --package-directory "$BITBUCKET_CLONE_DIR/artifacts"
            --package-file 'ORA-${TaskId}.${Version}.nupkg'
            --package-id "ORA-$STORY"
            --package-version "$BITBUCKET_BUILD_NUMBER"
            --package-author "Absolute Technology Ltd"
        artifacts:
          - artifacts/*.nupkg