NAV Navbar
Logo
Switch version:

Config Repositories

Config repository extension endpoint allows a plugin to externalize pipeline and environment configuration. This allows the configuration to be maintained and versioned outside of GoCD.

Getting started

Plugins in GoCD are implemented in Java and packaged as a JAR file.

Structure of a GoCD Plugin

A plugin for GoCD is a JAR file with the following structure:

plugin.jar
|
|-- plugin.xml
|-- com
|   \-- example
|       \-- go
|           \-- testplugin
|               \-- SimpleConfigRepositoryPlugin.class
|               \-- x.class
|               \-- y.class
\-- lib
    \-- dependency-1.jar
    \-- dependency-2.jar

The plugin jar is a self contained-jar. It is expected to contain the following inside it:

The plugin metadata plugin.xml

Here is an example plugin.xml:

<?xml version="1.0" encoding="utf-8" ?>
<!-- Your plugin id and version
     of this XML document, the
     version must be set to "1" -->
<go-plugin
  id="com.example.rocket.launcher"
  version="1">
  <about>
    <!-- The name of your plugin -->
    <name>Launches rockets</name>
    <!--  The version of your plugin -->
    <version>0.0.1</version>
    <!-- The minimum version of GoCD that this plugin requires -->
    <target-go-version>16.8.0</target-go-version>
    <!-- A longer description of your plugin  -->
    <description>Launches rockets, spaceships and other things to a destination of your choice.</description>

    <!-- Tell us who you are -->
    <vendor>
      <name>ACME Corp</name>
      <url>https://www.example.com</url>
    </vendor>

    <!-- If this plugin only supports certain OSes -->
    <target-os>
      <value>Linux</value>
      <value>Windows</value>
    </target-os>
  </about>
</go-plugin>

This is an XML file that should be present in the plugin jar file at the top level.

You can find the XML Schema for the plugins in the main repository.

The plugin extension class

Add this to your maven pom.xml:

<dependency>
  <groupId>cd.go.plugin</groupId>
  <artifactId>go-plugin-api</artifactId>
  <version>18.6.0</version>
</dependency>

In order to use the plugin extension class, you must add the following to your maven dependencies. If you’re using gradle, then use cd.go.plugin:go-plugin-api:18.6.0. You can find the latest version of the plugin in maven central.

An example plugin class implementation:

package com.example.go.testplugin;

import com.thoughtworks.go.plugin.api.*;
import com.thoughtworks.go.plugin.api.annotation.*;
import com.thoughtworks.go.plugin.api.exceptions.*;
import com.thoughtworks.go.plugin.api.request.*;
import com.thoughtworks.go.plugin.api.response.*;
import com.google.gson.Gson;
import java.util.*;

@Extension
public class SimpleConfigRepositoryPlugin implements GoPlugin {
  private GoApplicationAccessor accessor;

  // this method is executed once at startup
  public void initializeGoApplicationAccessor(GoApplicationAccessor accessor) {
    this.accessor = accessor;
  }

  // a GoPluginIdentifier tells GoCD what kind of a plugin this is
  // and what version(s) of the request/response API it supports
  public GoPluginIdentifier pluginIdentifier() {
    return new GoPluginIdentifier("configrepo", Arrays.asList("1.0"))
  }

  // handle the request and return a response
  // the response is very much like a HTTP response —
  // it has a status code, a response body and optional headers
  public GoPluginApiResponse handle(GoPluginApiRequest request) throws UnhandledRequestTypeException {
    if (request.requestName().equals("go.plugin-settings.get-view")) {
      String viewHtml = read(getClass().getResourceAsStream("/plugin-settings.template.html"));
      return new DefaultGoPluginApiResponse(200, viewHtml);
    if (request.requestName().equals("go.plugin-settings.validate-configuration")) {
      List errors = validate(request);
      return new DefaultGoPluginApiResponse(200, new Gson().toJson(errors));
    } else {
      throw new UnhandledRequestTypeException(request.requestName());
    }
  }
}

The plugin extension class is a Java class that implements the GoPlugin interface.

The other types in the example are:

Type Description
GoApplicationAccessor So that the plugin can make requests to the GoCD application to get additional information that is not provided by each request. For example, the plugin can ask for settings, credentials etc.
GoPluginIdentifier Provides information about the type of plugin and the version of the request/response it supports.
GoPluginApiRequest Represents the request message sent from GoCD to a plugin. The message will have a name and an optional JSON request body and the version of the extension.
GoPluginApiResponse Represents the response message as a result of processing the GoPluginApiRequest. Similar to GoPluginApiRequest, the response will have a status code and an optional JSON response body.

If you’re familiar with http request/responses handled by a web application, you will find this very familiar.

The plugin dependencies

Any dependencies that your plugin requires, should go into the lib directory inside the plugin JAR.

Requests from the GoCD server

In order to implement config repository extension point the following message must be implemented by the plugin.

These are general purpose messages that a plugin may implement to allow users to configure the plugin through the browser.

Parse directory

This message is a request to the plugin to parse a directory containing files which represent pipeline configuration. GoCD will periodically check the configured material repository (git, svn, etc) and if it finds a new commit, will send this request to the plugin, so that it can parse the configuration and tell it which pipelines have been defined in the external configuration.

Request name

parse-directory

Request body

Given the following config XML snippet —

<config-repos>
  <config-repo pluginId="your.plugin.id" id="some-unique-id">
    <git url="https://some.repo" />
    <configuration>
      <property>
        <key>some.key</key>
        <value>some.value</value>
      </property>
    </configuration>
  </config-repo>

The plugin will receive the following JSON body —

{
  "directory": "/absolute/path/to/directory/with/configs",
  "configurations": [
    {
      "key": "some.key",
      "value": "some.value"
    }
  ]
}

The request body will contain a JSON with an attribute directory, which points to a local checkout of the specified material repository. It will also contain an attribute configuration which contains the specified configuration. Note: This is not the configuration in the plugin settings page.

Response code

The plugin is expected to return status 200 if it can understand the request.

Response Body

An example response body of a problematic config:

{
  "target_version": 2,
  "environments": [],
  "pipelines": [],
  "errors": [
    {
      "location": "1.pipeline.json",
      "message": "There is an error here ..."
    }
  ]
}

A full response, with environments and pipelines:

{
  "target_version": 2,
  "environments": [
    {
      "name": "environmentName",
      "environment_variables": [
        {
          "name": "env1",
          "value": "val1"
        }
      ],
      "agents": [
        "agent-uuid-1",
        "agent-uuid-2"
      ],
      "pipelines": [
        "pipeline1"
      ]
    }
  ],
  "pipelines": [
    {
      "name": "pipeline1",
      "group": "group1",
      "label_template": "labelTemplate1",
      "lock_behavior": "lockOnFailure",
      "tracking_tool": {
        "link": "link",
        "regex": "regex",
        "location": "optional: key to group errors by"
      },
      "mingle": {
        "base_url": "baseUrl",
        "project_identifier": "projectId",
        "location": "optional: key to group errors by"
      },
      "timer": {
        "spec": "timerSpec",
        "only_on_changes": true,
        "location": "optional: key to group errors by"
      },
      "environment_variables": [
        {
          "name": "key1",
          "value": "value1",
          "location": "optional: key to group errors by"
        }
      ],
      "parameters": [
        {
          "name": "param1",
          "value": "value1",
          "location": "optional: key to group errors by"
        }
      ],
      "materials": [
        {
          "type": "git",
          "url": "gitrepo",
          "branch": "feature12",
          "shallow_clone": true,
          "filter": {
            "ignore": [
              "externals",
              "tools"
            ],
            "whitelist": []
          },
          "destination": "dir1",
          "auto_update": false,
          "name": "gitMaterial1",
          "location": "optional: key to group errors by"
        },
        {
          "type": "hg",
          "url": "repos/myhg",
          "filter": {
            "ignore": [
              "externals",
              "tools"
            ],
            "whitelist": []
          },
          "destination": "dir1",
          "auto_update": false,
          "name": "hgMaterial1",
          "location": "optional: key to group errors by"
        },
        {
          "type": "p4",
          "port": "10.18.3.102:1666",
          "username": "user1",
          "password": "pass1",
          "use_tickets": false,
          "view": "//depot/dev/src...          //anything/src/...",
          "filter": {
            "ignore": [
              "lib",
              "tools"
            ],
            "whitelist": []
          },
          "destination": "dir1",
          "auto_update": false,
          "name": "p4materialName",
          "location": "optional: key to group errors by"
        },
        {
          "type": "svn",
          "url": "http://svn",
          "username": "user1",
          "password": "pass1",
          "check_externals": true,
          "filter": {
            "ignore": [
              "tools",
              "lib"
            ],
            "whitelist": []
          },
          "destination": "destDir1",
          "auto_update": false,
          "name": "svnMaterial1",
          "location": "optional location for errors"
        },
        {
          "type": "tfs",
          "url": "url3",
          "username": "user4",
          "domain": "example.com",
          "password": "pass",
          "project": "projectDir",
          "filter": {
            "ignore": [
              "tools",
              "externals"
            ],
            "whitelist": []
          },
          "destination": "dir1",
          "auto_update": false,
          "name": "tfsMaterialName",
          "location": "optional: key to group errors by"
        },
        {
          "type": "package",
          "package_id": "package-id-in-bigger-config-xml",
          "name": "packageMaterialName",
          "location": "optional: key to group errors by"
        },
        {
          "type": "plugin",
          "scm_id": "someScmGitRepositoryId",
          "destination": "destinationDir",
          "filter": {
            "ignore": [
              "dir1",
              "dir2"
            ],
            "whitelist": []
          },
          "name": "myPluggableGit",
          "location": "optional: key to group errors by"
        },
        {
          "type": "dependency",
          "pipeline": "pipeline2",
          "stage": "build",
          "name": "pipe2",
          "location": "optional: key to group errors by"
        }
      ],
      "stages": [
        {
          "name": "stage1",
          "fetch_materials": true,
          "never_cleanup_artifacts": true,
          "clean_working_directory": true,
          "approval": {
            "type": "manual",
            "users": [
              "user1"
            ],
            "roles": [
              "role1"
            ],
            "location": "optional: key to group errors by"
          },
          "environment_variables": [
            {
              "name": "key1",
              "value": "value1",
              "location": "optional: key to group errors by"
            }
          ],
          "jobs": [
            {
              "name": "job1",
              "environment_variables": [
                {
                  "name": "key1",
                  "value": "value1",
                  "location": "optional: key to group errors by"
                }
              ],
              "tabs": [
                {
                  "name": "tab1",
                  "path": "path/to/file",
                  "location": "optional: key to group errors by"
                }
              ],
              "resources": [
                "resource1"
              ],
              "artifacts": [
                {
                  "source": "src1",
                  "destination": "dest1",
                  "type": "build",
                  "location": "optional: key to group errors by"
                },
                {
                  "source": "src2",
                  "destination": "dest2",
                  "type": "test",
                  "location": "optional: key to group errors by"
                }
              ],
              "properties": [
                {
                  "name": "name1",
                  "source": "target/file.xml",
                  "xpath": "string(//path/to/element)",
                  "location": "optional: key to group errors by"
                }
              ],
              "elastic_profile_id": "elasticProfile1",
              "run_instance_count": "all",
              "timeout": 20,
              "tasks": [
                {
                  "command": "command1",
                  "working_directory": "workDir1",
                  "timeout": 120,
                  "arguments": [
                    "arg1",
                    "arg2"
                  ],
                  "run_if": "passed",
                  "on_cancel": {
                    "plugin_configuration": {
                      "id": "pluginId",
                      "version": "version1",
                      "location": "optional: key to group errors by"
                    },
                    "configuration": [
                      {
                        "key": "key1",
                        "value": "value1",
                        "encrypted_value": "encryptedValue1",
                        "location": "optional: key to group errors by"
                      }
                    ],
                    "run_if": "passed",
                    "location": "optional: key to group errors by"
                  },
                  "location": "optional: key to group errors by"
                }
              ],
              "location": "optional: key to group errors by"
            }
          ],
          "location": "optional: key to group errors by"
        }
      ],
      "template": "template1",
      "location": "optional: key to group errors by"
    }
  ],
  "errors": []
}

The plugin is expected to return a JSON object (corresponding to ParseDirectoryResponseMessage with these top level attributes:

1. The target_version element

The target_version element should be set to 2 (static) for all new plugins. The change in the response based on changes in the value of target_version are described below:

Version change from … Change in schema
1 to 2 The property enable_pipeline_locking was changed to lock_behavior. The old values of true and false in enable_pipeline_locking were changed to lockOnFailure and none respectively in lock_behavior. A new value of unlockWhenFinished was introduced.

2. The environments element

The environments element is a list of environment objects (corresponding to CREnvironment) , each of which has a name, environment_variables (for the whole environment), agents in this environment and the pipelines in this environment.

See example shown.

3. The errors element

If this list has any errors defined (corresponding to CRError), the configuration will be considered invalid and the errors will be shown to the user in an appropriate manner.

4. The pipelines element

A fairly comprehensive response with a pipelines element is shown in the example. Not all of the parts are mandatory. Some noteworthy properties of this element are:

- This element corresponds to the CRPipeline object in GoCD.

- name is mandatory (and should be unique across the GoCD server).

- group is mandatory. It is the pipeline group that this pipeline should be added to.

- There should be at least one material and material names should be unique across the pipeline.

- There should either be stages defined or a template to fetch stages from. Not both.

- Environment variable names and parameter names should be unique across the pipeline.

- The different kinds of materials are shown in the example. They correspond to these objects in the GoCD contract part of the codebase: CRGitMaterial (git), CRSvnMaterial (SVN), CRP4Material (P4), CRHgMaterial (Hg), CRPackageMaterial (Package), CRTfsMaterial (TFS), CRDependencyMaterial (Dependency) and CRPluggableScmMaterial (SCM plugin).

- Jobs cannot have both resources and elastic_profile_id defined. They can have one of them or neither define.

- Most top-level attributes of a pipeline can have a location defined. If defined, that value will be used to show more meaningful error messages, if the GoCD server detects errors during its parsing and validation.

- See this note about the special material type: configrepo.

The special material type: configrepo

Given this config repo XML:

<config-repos>
   <config-repo pluginId="plugin.id" id="repo1">
     <git url="https://your/config/repo" branch="something" />
   </config-repo>
</config-repos>

Instead of using a material like this:

{
  "materials": [
    {
      "url": "https://your/config/repo",
      "branch": "something",
      "type": "git",
      "name": "mygit"
    }
  ]
}

… you can use this:

{
  "materials": [
    {
      "type": "configrepo",
      "name": "mygit"
    }
  ]
}

If the type of a material is configrepo, then the corresponding config repository’s information will be used as that material. This allows the pipeline definition to not have to repeat the config repository’s material information. This is most useful when the config repo definition and the rest of the source code are stored together.

The GoCD server clones the material configuration of the current repository (as is in XML) and replaces name, destination and filters (whitelist/blacklist). Then, it uses the modified clone in place of configrepo material.

Get Settings View

This is an optional message that the plugin may implement, should users want to configure the plugin from the GoCD admin page.

Request name

go.plugin-settings.get-view

Request body

The server will not provide a request body.

Response code

The plugin is expected to return status 200 if it can understand the request.

Response Body

A JSON settings view object

An example response body:

{
  "template": "<div>some html</div>"
}

Get Plugin Configuration

This is an optional message that the plugin may implement, should users want to configure the plugin from the GoCD admin page. This message allows the server to query a plugin about what properties are supported by this plugin.

Request name

go.plugin-settings.get-configuration

Request body

The server will not provide a request body.

Response code

The plugin is expected to return status 200 if it can understand the request.

Response Body

A JSON plugin settings configuration object.

Validate Plugin Configuration

If a plugin requires any configuration, this message must be implemented in order to validate the configuration.

Request name

go.plugin-settings.validate-configuration

Request body

An example validation request body

{
  "plugin-settings": {
      "server_url": {
        "value": "http://localhost.com"
      },
      "username": {
        "value": "user"
      },
      "password": {
        "value": "password"
      }
  }
}

The request body will contain a JSON with an attribute plugin-settings, which contains an object with the configuration keys and values that the plugin is expected to validate.

Response code

The plugin is expected to return status 200 if it can understand the request.

Response Body

The plugin should respond with JSON array response for each configuration key that has a validation error

[
  {
   "key": "server_url",
   "message": "Server URL cannot be localhost"
  }
]

If any of the input keys have a validation error on them, the plugin is expected to return a list of validation error objects. If the configuration is valid, the plugin should return an empty JSON array.

Requests to the GoCD server

The plugin may make the following requests to the server using GoApplicationAccessor#submit(GoApiRequest)

Get Plugin Settings

import com.thoughtworks.go.plugin.api.*;
import com.thoughtworks.go.plugin.api.annotation.Extension;
import com.thoughtworks.go.plugin.api.logging.Logger;
import com.thoughtworks.go.plugin.api.request.*;
import com.thoughtworks.go.plugin.api.response.*;
import com.google.gson.Gson;
import java.util.*;

@Extension
public class SimpleConfigRepositoryPlugin implements GoPlugin {
  private GoApplicationAccessor accessor;
  public static final Logger LOG = Logger.getLoggerFor(SimpleConfigRepositoryPlugin.class);

  public void initializeGoApplicationAccessor(GoApplicationAccessor accessor) {
    this.accessor = accessor;
  }

  public GoPluginIdentifier pluginIdentifier() {
    return new GoPluginIdentifier("configrepo", Arrays.asList("1.0"))
  }

  private PluginSettings getSettings() {
    Gson gson = new Gson();
    // create a request
    DefaultGoApiRequest request = new DefaultGoApiRequest(
      "go.processor.plugin-settings.get",
      "1.0",
      pluginIdentifier()
    );

    // set the request body
    Map<String, String> map = new HashMap<>();
    map.put("plugin-id", "com.example.rocket.launcher");
    request.setRequestBody(gson.toJson(map));

    // submit the request
    GoApiResponse response = accessor.submit(request);

    // check status
    if (response.responseCode() != 200) {
      LOG.error("The server sent an unexpected status code " + response.responseCode() + " with the response body " + response.responseBody());
    }

    // parse the response, using a json parser of your choice
    return gson.fromJson(response.responseBody(), PluginSettings.class);
  }
}

This messages allows a plugin to query the server to get the user configured settings for this plugin.

Request name

go.processor.plugin-settings.get

Request body

An example request body:

{
  "plugin-id": "sample-plugin-id"
}

Must be a JSON object with a key plugin-id with the value being the ID of your plugin.

Response code

The server is expected to return status 200 if it could process the request.

Response Body

An example response body:

{
  "server_url": "https://build.go.cd",
  "username": "view",
  "password": "password"
}

The server will send a map of settings.

Request/Response JSON Objects

The Settings View Object

Here’s an example of the settings view object:

{
  "template": "<div class=\"form_item_block\">...</div>"
}

Attribute Type Description
template String A string containing an HTML AngularJS based view.

This template is an AngularJS based template.

GoCD uses Angular JS as its template engine for the plugin UI. This allows plugin authors to use a limited set of AngularJS features to specify how the UI of their plugins looks.

Getting started with AngularJS based templates

Given a configuration:

<configuration>
  <property>
    <key>username</key>
    <value>alice</username>
  </property>
</configuration>

This gets converted into the following JSON representation in the browser:

{
  "username": "alice"
}

The AngularJS template is expected to bind to the JSON object shown above:

<div class="form_item_block">
  <label>Username:<span class='asterix'>*</span></label>
  <input ng-model="username" />
</div>

When an Angular template is used in a Go plugin, to define the configuration UI, the configuration key which is stored in the configuration XML is used everywhere and is expected to be consistent. Since Angular works off of JSON, GoCD will make sure that the key in the JSON provided to the Angular template is the same as the key in the configuration XML.

Suppose the key of the configuration property stored in the XML is “username”, with value, “alice”, then Go will make sure that the value is available to the template as “username” when used in an Angular-specific HTML attribute like “ng-model”.

Showing validation errors in the UI

We use some simple string replacement
to substitute GOINPUTNAME with a unique identifier
for your plugin in order to render
any server side errors

<div class="form_item_block">
  <label>Username:<span class='asterix'>*</span></label>
  <input ng-model="username" />
  <span class="form_error" ng-show="GOINPUTNAME[username].$error.server">
    {{ GOINPUTNAME[username].$error.server}}
  </span>
</div>

In case of validation errors returned by go.plugin-settings.validate-configuration, the error messages needs to be populated on the UI, use the snippet here to show the validation errors.

The Plugin Settings Configuration Object

Here’s an example of the plugin settings configuration object:

{
  "server_url": {
    "display-name": "Server URL",
    "display-order": "0"
  },
  "username": {
    "required": false,
    "display-name": "Username",
    "display-order": "1"
  },
  "password": {
    "secure": true,
    "required": false,
    "display-name": "Password",
    "display-order": "2"
  }
}

Attribute Type Description
display-name String The name of the property.
default-value String The default value of the property.
display-order String A string containing a numerical value.
required Boolean If the field is mandatory.
secure Boolean If the data in the field should be stored encrypted.

The Validation Error Object

Here’s an example of the validation error object:

[
  {
    "key": "email_address",
    "message": "Email address is invalid"
  },
  {
    "key": "password",
    "message": "Password must be provided"
  }
]

Attribute Type Description
key String The name of configuration key that has an error.
message String The error message associated with that key.