NAV Navbar
Logo
Switch version:

Artifact Plugins

Current Version

This document is for version 1.0 (the latest) of the artifacts endpoint.

Background

By default, GoCD internally manages and stores artifacts. The Artifact extension allows GoCD to make use of an external artifact store for storing artifacts while retaining traceability between pipelines. For example, by using the Docker artifact registry plugin, a user can publish an image to a docker registry, and subsequently fetch it in a downstream pipeline. Since GoCD is “aware” of the artifact store with this extension, it can still maintain the Value Stream between the upstream and downstream pipelines even if the artifact is stored outside of GoCD.

Developers can start building their own artifact plugins by forking the docker plugin as an example reference implementation.

How will it help you?

A feature like this will allow for artifacts to be uploaded and stored outside of GoCD. Previously, the artifacts were only managed on the GoCD server.

Requests from the GoCD server

In order to implement an artifact extension point the following messages must be implemented by the plugin.

These are general purpose messages that a plugin must implement to allow users to configure the plugin through the browser.

These are messages that a plugin must implement in order to allow users to configure artifact stores through the browser.

These are messages that a plugin must implement in order to allow users to configure publishing an artifact through the browser.

These are messages that a plugin must implement in order to allow users to configure fetching an artifact through the browser.

Publish Artifact

This message is a request to the plugin to publish an artifact to the specified artifact store.

Request name

cd.go.artifact.publish-artifact

Request body

Given the following config XML snippet —


<artifactStores>
      <artifactStore id="dockerhub" pluginId="cd.go.artifact.docker.registry">
        <property>
          <key>RegistryURL</key>
          <value>https://index.docker.io/v1/</value>
        </property>
        <property>
          <key>Username</key>
          <value>boohoo</value>
        </property>
        <property>
          <key>Password</key>
          <value>password</value>
        </property>
      </artifactStore>
    </artifactStores>
<pipelines group="first">
  <pipeline name="build">
    ...
    <job name="build-job">
      ...
      <artifacts>
        <artifact id="app-image" storeId="dockerhub">
          <configuration>
            <property>
              <key>Image</key>
              <value>gocd/gocd-demo</value>
            </property>
            <property>
              <key>Tag</key>
              <value>v${GO_PIPELINE_COUNTER}</value>
            </property>
          </configuration>
        </artifact>
      </artifacts>
    </job>
    ...
  </pipeline>
</pipelines>

The plugin will receive the following JSON body —

{
  "environment_variables":{
    "GO_PIPELINE_NAME":"build",
    "GO_TRIGGER_USER":"admin",
    "FOO": "bar"
  },
  "artifact_plan":{
    "configuration":{
      "BuildFile":"",
      "Image":"gocd/gocd-demo",
      "Tag":"v${GO_PIPELINE_COUNTER}"
    },
    "id":"app-image",
    "storeId":"dockerhub"
  },
  "artifact_store":{
    "configuration":{
      "RegistryURL":"https://index.docker.io/v1/",
      "Username":"boohoo",
      "Password":"password"
    },
    "id":"dockerhub"
  },
  "agent_working_directory":"/Users/varshavs/gocd/agent/pipelines/build"}

The request body will contain the following JSON elements:

Key Type Description
environment_variables Object A map of the environment variable that were available at the time the job was triggered.
artifact_plan Object This mainly contains the configuration properties that inform the plugin about the specific artifact that needs to be pushed.
artifact_store Object This section contains the details about the artifact store to which the plugin has to publish the artifact.
agent_working_directory String Since the artifact publishing happens on the go-agent side, the working directory is also passed along in the request in case the plugin wants to use it.

Response code

The plugin is expected to return status 200 if it can understand the request.

Response Body

The plugin is expected to return a json as shown. This json is written into a file called <plugin-id>.json on the agent and uploaded as a Build Artifact to the GoCD server to a directory called pluggable_metadata_folder

Plugin response body

{
  "metadata":{
    "image":"gocd/gocd-demo:v23",
    "digest":"sha256:f7840887b6f09f531935329a4ad1f6176866675873a8b3eed6a5894573da8247"
    }
}

The content of plugin-id.json file

{
  "app-image":{
  "image":"gocd/gocd-demo:v23",
  "digest":"sha256:f7840887b6f09f531935329a4ad1f6176866675873a8b3eed6a5894573da8247"
  }
}

Note: app-image is the artifact id specified in the job configuration

Fetch Artifact

Given the following config XML snippet —


<artifactStores>
      <artifactStore id="dockerhub" pluginId="cd.go.artifact.docker.registry">
        <property>
          <key>RegistryURL</key>
          <value>https://index.docker.io/v1/</value>
        </property>
        <property>
          <key>Username</key>
          <value>boohoo</value>
        </property>
        <property>
          <key>Password</key>
          <value>password</value>
        </property>
      </artifactStore>
    </artifactStores>
<pipelines group="first">
  <pipeline name="build">
    ...
    <job name="build-job">
      ...
      <artifacts>
        <artifact type="external" id="app-image" storeId="dockerhub">
          <configuration>
            <property>
              <key>Image</key>
              <value>gocd/gocd-demo</value>
            </property>
            <property>
              <key>Tag</key>
              <value>v${GO_PIPELINE_COUNTER}</value>
            </property>
          </configuration>
        </artifact>
      </artifacts>
    </job>
    ...

    <stage name="downstream_stage">
      ...
      <job name="downstream_job">
        <tasks>
            <fetchartifact artifactOrigin="external" stage="upstream_stage" job="build-job" artifactId="app-image">
              <configuration>
                <property>
                  <key>image_name</key>
                  <value>release-candidate</value>
                </property>
              </configuration>
            </fetchartifact>
        </tasks>
      </job>
      ...
    </stage>
  </pipeline>
</pipelines>

The plugin will receive the following JSON body —

{
  "fetch_artifact_configuration":{
    "image_name": "release-candidate"
  },
  "artifact_metadata":{
    "image":"gocd/gocd-demo:v23",
    "digest":"sha256:f7840887b6f09f531935329a4ad1f6176866675873a8b3eed6a5894573da8247"
  },
  "store_configuration":{
    "RegistryURL":"https://index.docker.io/v1/",
    "Username":"boohoo",
    "Password":"password"
  },
  "agent_working_directory":"/Users/varshavs/gocd/agent/pipelines/build"
}

This message is a request to the plugin to fetch an artifact from the specified artifact store.

Request name

cd.go.artifact.fetch-artifact

Request body

The request body will contain the following JSON elements:

Key Type Description
fetch_artifact_configuration Object This mainly contains the configuration properties that inform the plugin about the specific artifact that needs to be fetched.
artifact_metadata Object This is the information that was uploaded as a json file to the GoCD server at publish time. The json file is parsed and its contents are passed to the plugin at fetch time so that the plugin can fetch the right artifact.
store_configuration Object This section contains the details about the artifact store from which the plugin has to fetch the artifact.
agent_working_directory String Since the artifact download happens on the go-agent side, the working directory is also passed along in the request in case the plugin wants to use it.

Response code

The plugin is expected to return status 200 if it can understand the request.

Response Body

The plugin can respond with a list of environment variables which will be set on the job, for all further tasks in that job to use. It can also return an empty list ([]). Existing environment variables will be overridden, if they are returned as a part of this response.

Example response body:

[
  {
    "name": "IMAGE_ID",
    "value": "image1/v23",
    "secure": false
  },
  {
    "name": "MY_SECRET_ENV_VAR",
    "value": "some-password",
    "secure": true
  }
]

The response body will contain the following JSON elements:

Key Type Description
name String The key or name of the environment variable to set.
value String The value of the environment variable to set.
secure Boolean Should it be a secure environment variable or not?

Get Plugin Capabilities

This message is a request to the plugin to provide plugin capabilities. Currently the plugin does not support any capabilities, so plugins should return an empty JSON as a response. In the future, based on these capabilities GoCD would enable or disable the plugin features for a user.

Request name

cd.go.artifact.get-capabilities

Request body

Server sends request an Empty request body.

Response Body

The plugin is expected to return status 200 if it can understand the request, with an empty response body

Get Plugin Icon

This call is expected to return the icon for the plugin, so as to make it easy for users to identify the plugin.

Request name

cd.go.artifact.get-icon

Request body

The server will not provide a request body.

Response code

The plugin is expected to return status 200 if it can understand the request.

Response Body

An example plugin response body:

{
  "content_type": "image/svg+xml",
  "data": "PHN2ZyB2ZXJzaW9u..."
}

The plugin is expected to return an image object.

Get Store Config Metadata

This is a message that the plugin should implement, to allow users to configure global artifact stores from the Artifact Stores View in GoCD.

Request name

cd.go.artifact.store.get-metadata

Request body

The server will not provide a request body.

Response code

The plugin is expected to return status 200 if it can understand the request.

Response Body

An example response body for a docker plugin

[
  {
    "key": "RegistryURL",
    "metadata": {
      "required": true,
      "secure": false
    }
  },
  {
    "key": "Password",
    "metadata": {
      "required": true,
      "secure": true
    }
  }
]

A JSON metadata object.

Get Store Config View

This is a message that the plugin should implement, to allow users to configure artifact stores from the Artifact Stores View in GoCD.

Request name

cd.go.artifact.store.get-view

Request body

The server will not provide a request body.

Response code

The plugin is expected to return status 200 if it can understand the request.

Response Body

A JSON settings view object

An example response body:

{
  "template": "<div>some html</div>"
}

Validate Store Config

This call is expected to validate the user inputs that form a part of the artifact store.

Request name

cd.go.artifact.store.validate

Request body

The request body will contain a JSON object with the keys and values that form part of the profile.

An example validation request body

{
  "RegistryURL": "https://index.docker.io/v1/",
  "Username": "boohoo"
}

Response code

The plugin is expected to return status 200 if it can understand the request.

Response Body

The plugin should respond with JSON array response for each configuration key that has a validation error

[
  {
   "key": "RegistryURL",
   "message": "'RegistryURL is invalid'"
  }
]

If any of the input keys have a validation error on them, the plugin is expected to return a list of validation error objects. If the profile is valid, the plugin should return an empty JSON array.

Get Publish Artifact Metadata

This is a message that the plugin should implement, to allow users to configure external artifacts from the Job Config View in GoCD.

Request name

cd.go.artifact.publish.get-metadata

Request body

The server will not provide a request body.

Response code

The plugin is expected to return status 200 if it can understand the request.

Response Body

An example response body

[
  {
    "key": "Image",
    "metadata": {
      "required": true,
      "secure": false
    }
  },
  {
    "key": "Tag",
    "metadata": {
      "required": true,
      "secure": false
    }
  }
]

A JSON metadata object.

Get Publish Artifact View

This is a message that the plugin should implement, to allow users to configure external artifacts from the Job Config View in GoCD.

Request name

cd.go.artifact.publish.get-view

Request body

The server will not provide a request body.

Response code

The plugin is expected to return status 200 if it can understand the request.

Response Body

A JSON settings view object

An example response body:

{
  "template": "<div>some html</div>"
}

Validate Publish Artifact Config

This call is expected to validate the user inputs that form a part of the publish external artifact config.

Request name

cd.go.artifact.publish.validate

Request body

The request body will contain a JSON object with the keys and values that form part of the profile.

An example validation request body

{
  "Image": "gocd/gocd-demo",
  "Tag": "boohoo",
  "BuildFile": "foo.json"
}

Response code

The plugin is expected to return status 200 if it can understand the request.

Response Body

The plugin should respond with JSON array response for each configuration key that has a validation error

[
  {
   "key": "Image",
   "message": "Either `BuildFile` or `Image` must be specified"
  },
  {
   "key": "BuildFile",
   "message": "Either `BuildFile` or `Image` must be specified"
  }
]

If any of the input keys have a validation error on them, the plugin is expected to return a list of validation error objects. If the profile is valid, the plugin should return an empty JSON array.

Get Fetch Artifact Metadata

This is a message that the plugin should implement, to allow users to configure fetch artifact task from the Task View in GoCD.

Request name

cd.go.artifact.fetch.get-metadata

Request body

The server will not provide a request body.

Response code

The plugin is expected to return status 200 if it can understand the request.

Response Body

An example response body:

[
  {
    "key": "DestOnAgent",
    "metadata": {
      "required": true,
      "secure": false
    }
  }
]

A JSON metadata object.

Get Fetch Artifact View

This is a message that the plugin should implement, to allow users to configure fetch artifact task from the Task View in GoCD.

Request name

cd.go.artifact.fetch.get-view

Request body

The server will not provide a request body.

Response code

The plugin is expected to return status 200 if it can understand the request.

Response Body

A JSON settings view object

An example response body:

{
  "template": "<div>some html</div>"
}

Validate Fetch Artifact Config

This call is expected to validate the user inputs that form a part of the fetch external artifact config.

Request name

cd.go.artifact.publish.validate

Request body

The request body will contain a JSON object with the keys and values that form part of the profile.

An example validation request body

{
  "Dest": "foo"
}

Response code

The plugin is expected to return status 200 if it can understand the request.

Response Body

The plugin should respond with JSON array response for each configuration key that has a validation error

[
  {
   "key": "Dest",
   "message": "Invalid"
  }
]

If any of the input keys have a validation error on them, the plugin is expected to return a list of validation error objects. If the profile is valid, the plugin should return an empty JSON array.

Requests to the GoCD server

The plugin may make the following requests to the server using GoApplicationAccessor#submit(GoApiRequest)

Send Console Log

The plugin can send console logs to the server while fetching or publishing artifacts.

Request name

go.processor.artifact.console-log

Request body

The request body must contain the logLevel and a message string. The logLevel must be INFO or ERROR.

Example Request Body

{
  "logLevel": "INFO",
  "message" : "This is an info message."
}

Response code

The server is expected to return status 200 if it could process the request.

Response body

The server will not send a response body.

Console Logger Example

import com.google.gson.Gson;
import com.thoughtworks.go.plugin.api.GoApplicationAccessor;
import com.thoughtworks.go.plugin.api.GoPluginIdentifier;
import com.thoughtworks.go.plugin.api.logging.Logger;
import com.thoughtworks.go.plugin.api.request.DefaultGoApiRequest;
import com.thoughtworks.go.plugin.api.response.DefaultGoApiResponse;
import com.thoughtworks.go.plugin.api.response.GoApiResponse;

import java.util.Collections;

public class ConsoleLogger {
    private static final String SEND_CONSOLE_LOG = "go.processor.artifact.console-log";
    private static final String API_VERSION = "1.0";
    private GoPluginIdentifier PLUGIN_IDENTIFIER = new GoPluginIdentifier("artifact", Collections.singletonList(API_VERSION));
    private static final Logger LOG = Logger.getLoggerFor(DockerRegistryArtifactPlugin.class);
    private static ConsoleLogger consoleLogger;
    private final GoApplicationAccessor accessor;

    private ConsoleLogger(GoApplicationAccessor accessor) {
        this.accessor = accessor;
    }

    public void info(String message) {
        sendLog(new ConsoleLogMessage(ConsoleLogMessage.LogLevel.INFO, message));
    }

    public void error(String message) {
        sendLog(new ConsoleLogMessage(ConsoleLogMessage.LogLevel.ERROR, message));
    }

    private void sendLog(ConsoleLogMessage consoleLogMessage) {
        DefaultGoApiRequest request = new DefaultGoApiRequest(SEND_CONSOLE_LOG, API_VERSION, PLUGIN_IDENTIFIER);
        request.setRequestBody(consoleLogMessage.toJSON());

        GoApiResponse response = accessor.submit(request);
        if (response.responseCode() != DefaultGoApiResponse.SUCCESS_RESPONSE_CODE) {
            LOG.error(String.format("Failed to submit console log: %s", response.responseBody()));
        }
    }

    public static ConsoleLogger getLogger(GoApplicationAccessor accessor) {
        if (consoleLogger == null) {
            synchronized (ConsoleLogger.class) {
                if (consoleLogger == null) {
                    consoleLogger = new ConsoleLogger(accessor);
                }
            }
        }

        return consoleLogger;
    }

    static class ConsoleLogMessage {
        private LogLevel logLevel;
        private String message;

        public ConsoleLogMessage(LogLevel logLevel, String message) {
            this.message = message;
            this.logLevel = logLevel;
        }

        public String toJSON() {
            return new Gson().toJson(this);
        }

        enum LogLevel {
            INFO, ERROR
        }
    }
}

Request/Response JSON Objects

The Settings View Object

Here’s an example of the settings view object:

{
  "template": "<div class=\"form_item_block\">...</div>"
}

Attribute Type Description
template String A string containing an HTML AngularJS based view.

This template is an AngularJS based template.

GoCD uses Angular JS as its template engine for the plugin UI. This allows plugin authors to use a limited set of AngularJS features to specify how the UI of their plugins looks.

Getting started with AngularJS based templates

Given a configuration:

<configuration>
  <property>
    <key>username</key>
    <value>alice</username>
  </property>
</configuration>

This gets converted into the following JSON representation in the browser:

{
  "username": "alice"
}

The AngularJS template is expected to bind to the JSON object shown above:

<div class="form_item_block">
  <label>Username:<span class='asterix'>*</span></label>
  <input ng-model="username" />
</div>

When an Angular template is used in a Go plugin, to define the configuration UI, the configuration key which is stored in the configuration XML is used everywhere and is expected to be consistent. Since Angular works off of JSON, GoCD will make sure that the key in the JSON provided to the Angular template is the same as the key in the configuration XML.

Suppose the key of the configuration property stored in the XML is “username”, with value, “alice”, then Go will make sure that the value is available to the template as “username” when used in an Angular-specific HTML attribute like “ng-model”.

Showing validation errors in the UI

We use some simple string replacement
to substitute GOINPUTNAME with a unique identifier
for your plugin in order to render
any server side errors

<div class="form_item_block">
  <label>Username:<span class='asterix'>*</span></label>
  <input ng-model="username" />
  <span class="form_error" ng-show="GOINPUTNAME[username].$error.server">
    {{ GOINPUTNAME[username].$error.server}}
  </span>
</div>

In case of validation errors returned by go.plugin-settings.validate-configuration, the error messages needs to be populated on the UI, use the snippet here to show the validation errors.

The Plugin Settings Configuration Object

Here’s an example of the plugin settings configuration object:

{
  "server_url": {
    "display-name": "Server URL",
    "display-order": "0"
  },
  "username": {
    "required": false,
    "display-name": "Username",
    "display-order": "1"
  },
  "password": {
    "secure": true,
    "required": false,
    "display-name": "Password",
    "display-order": "2"
  }
}

Attribute Type Description
display-name String The name of the property.
default-value String The default value of the property.
display-order String A string containing a numerical value.
required Boolean If the field is mandatory.
secure Boolean If the data in the field should be stored encrypted.

The Validation Error Object

Here’s an example of the validation error object:

[
  {
    "key": "email_address",
    "message": "Email address is invalid"
  },
  {
    "key": "password",
    "message": "Password must be provided"
  }
]

Attribute Type Description
key String The name of configuration key that has an error.
message String The error message associated with that key.

The Image Object

Here’s an example of the image object:

{
  "content_type": "image/svg+xml",
  "data": "...."
}

Attribute Type Description
content_type String A valid content type for the image. Please make sure the content type is supported by most browsers.
data String A base-64 encoded (single-line non-chunking) byte array of the byte-sequence that composes the image.

The Metadata Object

Here’s an example of the metadata object:

{
  "key": {
    "required": "true",
    "secure": "true"
  }
}

Attribute Type Description
key String The name of the configuration property supported by an artifact store, publish artifact config and fetch artifact config.
metadata Object The metadata associated with the key. Valid keys are required and secure.

The server info object

Here’s an example of the server info object:

{
  "server_id": "df0cb9be-2696-4689-8d46-1ef3c4e4447c",
  "site_url": "http://example.com:8153/go",
  "secure_site_url": "https://example.com:8154/go"
}

Attribute Type Description
server_id String This contains a unique identifier for this server.
site_url String This contains the site url configured for this server.
secure_site_url String This contains the secure site url configured for this server.