1. dxpy Package

This package includes three key modules:

• dxpy.bindings: Contains useful Pythonic bindings for interacting with remote objects via the DNAnexus API server. For convenience, this is automatically imported directly into the namespace under dxpy when dxpy is imported.

• dxpy.api: Contains low-level wrappers that can be called directly to make the respective API calls to the API server.

• dxpy.exceptions: Contains exceptions used in the other dxpy modules.

1.1. Package Configuration

When this package is imported, configuration values will be loaded from the following sources in order of decreasing priority:

1. Environment variables

2. Values stored in ~/.dnanexus_config/environment

3. Hardcoded defaults

The bindings are configured by the following environment variables:

DX_SECURITY_CONTEXT

A JSON hash containing your auth token, typically of the form {"auth_token_type": "Bearer", "auth_token": "YOUR_TOKEN"}.

DX_APISERVER_PROTOCOL

Either http or https (usually https).

DX_APISERVER_HOST

Hostname of the DNAnexus API server.

DX_APISERVER_PORT

Port of the DNAnexus API server.

DX_JOB_ID

Should only be present if run in an Execution Environment; indicates the ID of the currently running job.

DX_WORKSPACE_ID

Should only be present if run in an Execution Environment; indicates the running job’s temporary workspace ID.

DX_PROJECT_CONTEXT_ID

Indicates either the project context of a running job, or the default project to use for a user accessing the platform from the outside.

The following fields can be used to read the current configuration values:

dxpy.APISERVER_PROTOCOL

Protocol being used to access the DNAnexus API server. Either http or https (usually https).

dxpy.APISERVER_HOST

Hostname of the DNAnexus API server.

dxpy.APISERVER_PORT

Port of the DNAnexus API server.

dxpy.JOB_ID

Indicates the ID of the currently running job, or None if we are not in an Execution Environment.

dxpy.WORKSPACE_ID

Indicates the temporary workspace ID of the currently running job, or the current project if we are not in an Execution Environment.

dxpy.PROJECT_CONTEXT_ID

Indicates either the project context of a running job, if there is one, or the default project that is being used, for users accessing the platform from the outside.

dxpy.USER_AGENT

The user agent string that dxpy will send to the server with each request.

The dxpy.DXHTTPRequest() function uses the DX_SECURITY_CONTEXT and DX_APISERVER_* variables to select an API server and provide appropriate authentication headers to it. (Note: all methods in the dxpy.api module, and by extension any of the bindings methods that make API calls, use this function.)

All object handler methods that require a project or data container ID use by default the DX_WORKSPACE_ID (if running inside an Execution Environment) or DX_PROJECT_CONTEXT_ID (otherwise).

The following functions can be used to override any of the settings obtained from the environment for the duration of the session:

• dxpy.set_security_context(): to specify an authentication token

• dxpy.set_api_server_info(): to specify the API server (host, port, or protocol)

• dxpy.set_workspace_id(): to specify the default data container

To pass API server requests through an HTTP(S) proxy, set the following environment variables:

HTTP_PROXY

HTTP proxy, in the form ‘protocol://hostname:port’ (e.g. ‘http://10.10.1.10:3128’)

HTTPS_PROXY

HTTPS proxy, in the form ‘protocol://hostname:port’

dxpy.configure_urllib3()

dxpy.DXHTTPRequest(resource, data, method='POST', headers=None, auth=True, timeout=905, use_compression=None, jsonify_data=True, want_full_response=False, decode_response_body=True, prepend_srv=True, session_handler=None, max_retries=6, always_retry=False, **kwargs)

Parameters:

• resource (string) – API server route, e.g. “/record/new”. If prepend_srv is False, a fully qualified URL is expected. If this argument is a callable, it will be called just before each request attempt, and expected to return a tuple (URL, headers). Headers returned by the callback are updated with headers (including headers set by this method).

• data (list or dict, if jsonify_data is True; or string or file-like object, otherwise) – Content of the request body

• headers (dict) – Names and values of HTTP headers to submit with the request (in addition to those needed for authentication, compression, or other options specified with the call).

• auth (tuple, object, True (default), or None) – Controls the Authentication header or other means of authentication supplied with the request. If True (default), a token is obtained from the DX_SECURITY_CONTEXT. If the value evaluates to false, no action is taken to prepare authentication for the request. Otherwise, the value is assumed to be callable, and called with three arguments (method, url, headers) and expected to prepare the authentication headers by reference.

• timeout (float) – HTTP request timeout, in seconds

• use_compression (string or None) – Deprecated

• jsonify_data (boolean) – If True, data is converted from a Python list or dict to a JSON string

• want_full_response (boolean) – If True, the full urllib3.response.HTTPResponse object is returned (otherwise, only the content of the response body is returned)

• decode_response_body (boolean) – If True (and want_full_response is False), the response body is decoded and, if it is a JSON string, deserialized. Otherwise, the response body is uncompressed if transport compression is on, and returned raw.

• prepend_srv (boolean) – If True, prepends the API server location to the URL

• session_handler – Deprecated.

• max_retries (int) –

  Maximum number of retries to perform for a request. A “failed” request is retried if any of the following is true:

  • A response is received from the server, and the content length received does not match the “Content-Length” header.

  • A response is received from the server, and the response has an HTTP status code in 5xx range.

  • A response is received from the server, the “Content-Length” header is not set, and the response JSON cannot be parsed.

  • No response is received from the server, and either always_retry is True or the request method is “GET”.

• always_retry (boolean) –

  If True, indicates that it is safe to retry a request on failure

  • Note: It is not guaranteed that the request will always be retried on failure; rather, this is an indication to the function that it would be safe to do so.

Returns:

Response from API server in the format indicated by want_full_response and decode_response_body.

Raises:

exceptions.DXAPIError or a subclass if the server returned a non-200 status code; urllib3.exceptions.HTTPError if an invalid response was received from the server; or urllib3.exceptions.ConnectionError if a connection cannot be established.

Wrapper around urllib3.request() that makes an HTTP request, inserting authentication headers and (by default) converting data to JSON.

Note

Bindings methods that make API calls make the underlying HTTP request(s) using DXHTTPRequest(), and most of them will pass any unrecognized keyword arguments you have supplied through to DXHTTPRequest().

class dxpy.DXHTTPOAuth2(security_context)

dxpy.set_api_server_info(host=None, port=None, protocol=None)

Parameters:

• host (string) – API server hostname

• port (string) – API server port. If not specified, port is guessed based on protocol.

• protocol (string) – Either “http” or “https”

Overrides the current settings for which API server to communicate with. Any parameters that are not explicitly specified are not overridden.

dxpy.set_security_context(security_context)

Parameters:

security_context (dict) – Authentication hash, usually with keys auth_token_type set to Bearer and auth_token set to the authentication token.

Sets the security context to use the provided token.

dxpy.set_job_id(dxid)

Parameters:

dxid (string) – ID of a job

Sets the ID of the running job.

Warning

This function is only really useful if you are developing code that will run in and interact with the Execution Environment, but wish to test it outside of an actual Execution Environment.

dxpy.set_workspace_id(dxid)

Parameters:

dxid (string) – ID of a project or workspace

Sets the default data container for object creation and modification to the specified project or workspace.

dxpy.set_project_context(dxid)

Parameters:

dxid (string) – Project ID

Sets the project context for a running job.

Warning

This function is only really useful if you are developing code that will run in and interact with the Execution Environment but wish to test it outside of an actual Execution Environment.

It does not change the default data container in which new objects are created or name resolution is performed. If you want to do that, use set_workspace_id() instead.

dxpy.set_watch_port(port=None)

Parameters:

port (string) – port to use for streaming job logs

Sets the port to use for streaming job logs via dx watch inside the Execution Environment

Warning

This function is only really useful if you are developing code that will run in and interact with the Execution Environment.

dxpy.get_auth_server_name(host_override=None, port_override=None, protocol='https')

Chooses the auth server name from the currently configured API server name.

Raises DXError if the auth server name cannot be guessed and the overrides are not provided (or improperly provided).

dxpy.append_underlying_workflow_describe(globalworkflow_desc)

Adds the “workflowDescribe” field to the config for each region of the global workflow. The value is the description of an underlying workflow in that region.