Clipper API API Reference

REST API for instrumenting with Clipper.

There are two frontend avaliable for query.

  • In admin frontend, you can manage your applications and models.
  • In query frontend, you can send query to depolyed models.

The admin address can be obtained from querying ClipperConnection.cm.get_admin_addr() .

The query address can be obtained from querying ClipperConnection.cm.get_query_addr() .

Request Content-Types: application/json
Response Content-Types: application/json
Version: 0.2.0

admin_addr

Admin Address

add_app

POST /admin/add_app

Register a new application with Clipper.

An application in Clipper corresponds to a named REST endpoint that can be used to request predictions. This command will attempt to create a new endpoint with the provided name. Application names must be unique. This command will fail if an application with the provided name already exists.

Request Example
{
  "default_output": "[42.0]",
  "input_type": "doubles",
  "latency_slo_micros": 100000,
  "name": "simple-app"
}
200 OK

OK

400 Bad Request

Bad Request

add_model

POST /admin/add_model

Registers a new model version with Clipper.

This method does not launch any model containers, it only registers the model description (metadata such as name, version, and input type) with Clipper. A model must be registered with Clipper before it can be linked to an application.

You should rarely have to use this method directly. Using one the Clipper deployer methods in clipper_admin.deployers or calling build_and_deploy_model or deploy_model will automatically register your model with Clipper.

Request Example
{
  "batch_size": 10,
  "container_name": "clipper/example-container",
  "input_type": "doubles",
  "labels": [
    "Team: DevOps"
  ],
  "model_name": "sum-model",
  "model_version": "1"
}
200 OK

OK

400 Bad Request

Bad Request

add_model_links

POST /admin/add_model_links

Routes requests from the specified app to be evaluted by the specified model.

Note:

Both the specified model and application must be registered with Clipper, and they must have the same input type. If the application has previously been linked to a different model, this command will fail.

Request Example
{
  "app_name": "sum-application",
  "model_names": [
    "basic-sum-1"
  ]
}
200 OK

OK

400 Bad Request

Bad Request

get_all_applications

POST /admin/get_all_applications

Gets information about all applications registered with Clipper.

If 'verbose' is set to False, the returned list contains the apps' names; if 'verbose' set to True, the list contains application info dictionaries. These dictionaries have the same attribute name-value pairs that were provided to clipper_admin.ClipperConnection.register_application .

Returns a list of information about all apps registered to Clipper. If no apps are registered with Clipper, an empty list is returned.

Request Example
{
  "verbose": "true"
}

Success

400 Bad Request

Bad Request

Response Example (200 OK)
[
  {
    "default_output": "[42.0]",
    "input_type": "doubles",
    "latency_slo_micros": 100000,
    "name": "simple-app"
  }
]

get_all_containers

POST /admin/get_all_containers

Gets information about all model containers registered with Clipper.

If 'verbose' is set to False, the returned list contains the apps' names. If 'verbose' is set to True, the list contains container info dictionaries.

Returns a list of information about all model containers known to Clipper. If no containers are registered with Clipper, an empty list is returned.

Request Example
{
  "verbose": "true"
}

Success

400 Bad Request

Bad Request

Response Example (200 OK)
[
  {
    "input_type": "doubles",
    "model_id": "simple-example:1",
    "model_name": "simple-example",
    "model_replica_id": 1,
    "model_version": "1"
  }
]

get_all_models

POST /admin/get_all_models

Gets information about all models registered with Clipper.

If 'verbose' is set to False, the returned list contains the models' names. If 'verbose' set to True, the list contains model info dictionaries.

Returns a list of information about all apps registered to Clipper. If no models are registered with Clipper, an empty list is returned.

Request Example
{
  "verbose": "true"
}
200 OK

Success

400 Bad Request

Bad Request

Response Example (200 OK)
[
  {
    "batch_size": 10,
    "container_name": "clipper/example-container",
    "input_type": "doubles",
    "labels": [
      "Team: DevOps"
    ],
    "model_name": "sum-model",
    "model_version": "1"
  }
]

get_application

POST /admin/get_application

Gets detailed information about a registered application.

'name' parameter is the name of the application to look up.

Returns a dictionary with the specified application's info. This will contain the attribute name-value pairs that were provided to clipper_admin.ClipperConnection.register_application . If no application with name name is registered with Clipper, None is returned.

Request Example
{
  "name": "simple-example"
}

Success

400 Bad Request

Bad Request

Response Example (200 OK)
{
  "default_output": "[42.0]",
  "input_type": "doubles",
  "latency_slo_micros": 100000,
  "name": "simple-app"
}

get_container

POST /admin/get_container

Gets detailed information about a registered container.

'name' is the name of the container to look up

'version' is the version of the container to look up

'replica_id' is the container replica to look up

Returns a dictionary with the specified container's info. If no corresponding container is registered with Clipper, None is returned.

Request Example
{
  "input_type": "doubles",
  "model_id": "simple-example:1",
  "model_name": "simple-example",
  "model_replica_id": 1,
  "model_version": "1"
}

Success

400 Bad Request

Bad Request

Response Example (200 OK)
{
  "input_type": "doubles",
  "model_id": "simple-example:1",
  "model_name": "simple-example",
  "model_replica_id": 1,
  "model_version": "1"
}

get_linked_models

POST /admin/get_linked_models

Retrieves the models linked to the specified application.

'app_name' parameter is the name of the application

Returns a list of the names of models linked to the app. If no models are linked to the specified app, None is returned.

Request Example
{
  "app_name": "simple-example"
}
200 OK

Success

400 Bad Request

Bad Request

Response Example (200 OK)
[
  "Model Name"
]

get_model

POST /admin/get_model

Gets detailed information about a registered model.

'model_name' is the name of the model to look up

'model_version' is the version of the model to look up

Returns a dictionary with the specified model's info. If no model with name model_name@model_version is registered with Clipper, None is returned.

Request Example
{
  "model_name": "sum_model",
  "model_version": "1"
}
200 OK

Success

400 Bad Request

Bad Request

Response Example (200 OK)
{
  "batch_size": 10,
  "container_name": "clipper/example-container",
  "input_type": "doubles",
  "labels": [
    "Team: DevOps"
  ],
  "model_name": "sum-model",
  "model_version": "1"
}

set_model_version

POST /admin/set_model_version

Changes the current model version to "model_version".

This method can be used to perform model roll-back and roll-forward. The version can be set to any previously deployed version of the model.

Note:

Model versions automatically get updated when clipper_admin.ClipperConnection.deploy_model() is called. There is no need to manually update the version after deploying a new model.

Request Example
{
  "model_name": "sum_model",
  "model_version": "1"
}
200 OK

OK

400 Bad Request

Bad Request

query_addr

predict

POST /query_addr/{application_name}/predict

Submit a new query to application. The application name needs to specified in the path; and data needs to be submitted via a POST JSON request.

Response will contains either the data returned by model or the default response. This can be distinguished by the default field. If the default field is True, there will be another field present called "default_explanation" whose value is a string describing the reason that a default response was returned. This field is not present if default is False.

application_name

The name of appplication to query.

type
string
in
path
Request Example
{
  "input": [
    2.3
  ]
}

Success

400 Bad Request

Bad Request

Response Example (200 OK)
{
  "default": "true",
  "default_explanation": "Failed to retrieve a prediction response within the specified latency SLO",
  "output": 2.2,
  "query_id": 42
}

Schema Definitions

App Name: object

app_name: string

Exact name of app

Example
{
  "app_name": "simple-example"
}

Application: object

default_output: string

The default output for the application. The default output will be returned whenever an application is unable to receive a response from a model within the specified query latency SLO (service level objective). The reason the default output was returned is always provided as part of the prediction response object.

input_type: string integers , floats , doubles , bytes , strings

The type of the request data this endpoint can process. Input type can be one of "integers", "floats", "doubles", "bytes", or "strings".

latency_slo_micros: integer

The query latency objective for the application in microseconds. This is the processing latency between Clipper receiving a request and sending a response. It does not account for network latencies before a request is received or after a response is sent. If Clipper cannot process a query within the latency objective, the default output is returned. Therefore, it is recommended that he SLO not be set aggressively low unless absolutely necessary. 100000 (100ms) is a good starting value, but the optimal latency objective will vary depending on the application.

name: string

The unique name of the application.

Example
{
  "default_output": "[42.0]",
  "input_type": "doubles",
  "latency_slo_micros": 100000,
  "name": "simple-app"
}

Model: object

batch_size: integer

The user-defined query batch size for the model. Replicas of the model will attempt to process at most batch_size queries simultaneously. They may process smaller batches if batch_size queries are not immediately available. If the default value of -1 is used, Clipper will adaptively calculate the batch size for individual replicas of this model.

container_name: string

A docker image name. If provided, the image will be recorded as part of the model descrtipin in Clipper when registering the model but this method will make no attempt to launch any containers with this image.

input_type: string integers , floats , doubles , bytes , strings

The type of the request data this endpoint can process. Input type can be one of "integers", "floats", "doubles", "bytes", or "strings". See the http://clipper.ai/tutorials/basic_concepts/#input-types for more details on picking the right input type for your application.

labels: string[]

A list of strings annotating the model. These are ignored by Clipper and used purely for user annotations.

model_name: string

The name of the deployed model. The model name must be valid DNS-1123 subdomains. Each must consist of lower case alphanumeric characters, ‘-‘ or ‘.’, and must start and end with an alphanumeric character (e.g. ‘example.com’, regex used for validation is [a-z0-9]([-a-z0-9]*[a-z0-9])?Z .

model_version: string

The version to assign this model. Versions must be unique on a per-model basis, but may be re-used across different models. The model version must be valid DNS-1123 subdomains. Each must consist of lower case alphanumeric characters, ‘-‘ or ‘.’, and must start and end with an alphanumeric character (e.g. ‘example.com’, regex used for validation is [a-z0-9]([-a-z0-9]*[a-z0-9])?Z .

Example
{
  "batch_size": 10,
  "container_name": "clipper/example-container",
  "input_type": "doubles",
  "labels": [
    "Team: DevOps"
  ],
  "model_name": "sum-model",
  "model_version": "1"
}

Model Container: object

input_type: string integers , floats , doubles , bytes , strings

The type of the request data this endpoint can process. Input type can be one of "integers", "floats", "doubles", "bytes", or "strings". See the http://clipper.ai/tutorials/basic_concepts/#input-types for more details on picking the right input type for your application.

model_id: string

Automatically assigned model id

model_name: string

The name of the deployed model. The model name must be valid DNS-1123 subdomains. Each must consist of lower case alphanumeric characters, ‘-‘ or ‘.’, and must start and end with an alphanumeric character (e.g. ‘example.com’, regex used for validation is [a-z0-9]([-a-z0-9]*[a-z0-9])?Z .

model_replica_id: integer

Automatically Assgined Replica ID

model_version: string

The version to assign this model. Versions must be unique on a per-model basis, but may be re-used across different models. The model version must be valid DNS-1123 subdomains. Each must consist of lower case alphanumeric characters, ‘-‘ or ‘.’, and must start and end with an alphanumeric character (e.g. ‘example.com’, regex used for validation is [a-z0-9]([-a-z0-9]*[a-z0-9])?Z .

Example
{
  "input_type": "doubles",
  "model_id": "simple-example:1",
  "model_name": "simple-example",
  "model_replica_id": 1,
  "model_version": "1"
}

Model Version: object

model_name: string

The name of the model

model_version: string

str | obj with str representation. The version of the model. Note that version must be a model version that has already been deployed.

Example
{
  "model_name": "sum_model",
  "model_version": "1"
}

Name: object

name: string

Exact name of app

Example
{
  "name": "simple-example"
}

Query Input: object

input: object[]

The query input as list.

Example
{
  "input": [
    2.3
  ]
}

Query Result: object

default: boolean

Whether or not this result is default response. The reason for default is given in the default_explanation field.

default_explanation: string

Explanation for returning the default response. This field is only present when default is true.

output: object

The prediction output.

query_id: integer

The query id by application

Example
{
  "default": "true",
  "default_explanation": "Failed to retrieve a prediction response within the specified latency SLO",
  "output": 2.2,
  "query_id": 42
}

Verbosity: object

verbose: boolean

If False, only necessary values like names will be returned; if True, more information will be provided.

Example
{
  "verbose": "true"
}