A class runs single threaded benchmark.

Abstract benchmark class.

A class runs single threaded benchmark.

A class that runs a benchmark using the WorkLoadManager .

DsEngineProvider is the DeepSpeed implementation of EngineProvider .

PyEngineProvider is the Python implementation of EngineProvider .

The PyEngine is an implementation of the Engine that runs Python worker.

PyModel is the Python engine implementation of Model .

A special io.netty.channel.ChannelInboundHandler which offers an easy way to initial...

A helper for creating a CacheEngine .

A cache that can be used for streaming online caching, streaming pagination, or asyn...

A CacheEngine that stores elements in DynamoDB.

A CacheEngine that stores elements in working memory. Note that this is not suitable...

A CacheEngine that stores elements in S3.

A plugin handles caching options.

Interface to be implemented by HtttpRequestHandler. Classes implementing this interf...

A class handling inbound HTTP requests for the console API.

A class handling inbound HTTP requests for the KServe API.

A HTTP endpoint handler to return information about the loaded plugins. This class i...

A class handles static file in "public" folder or resources classpath. this handler ...

A class handling inbound HTTP requests to the management API for adapters.

A class handling inbound HTTP requests.

HttpRequestHandler that tries to process a http- request using the configured Request...

A class handling inbound HTTP requests for the management API.

A class handling unhandled inbound HTTP requests.

A class handling inbound HTTP requests to the management API.

The PluginManager is responsible to load and manage plugins from the file system. Th...

The Plugin Manager is responsible to load and manage plugins from the filesystem. Th...

Reads plugin-metadata from plugin.definition file which have to be in property file ...

Reads plugin-metadata from an url and parse the content. Implementations typically r...

A class represent a loaded model and it's metadata.

A WorkerPoolConfig represents a task that could be run in the WorkLoadManager . Each...

a batch aggregator that never terminates by itself. the surrounding thread has to be...

abstract class for all BatchAggregators. A batch aggregator check working queue and ...

a batch aggregator that terminates after a maximum idle time.

An overload of Adapter for the python engine.

An adapter is a modification producing a variation of a model that can be used durin...

Thrown to throttle when a job is run but the job queue capacity is exceeded.

Thrown when an exception occurs inside the ai.djl. serving.wlm.WorkLoadManager .

Thrown when no enough memory to load the model.

Thrown when a job is run but all workers are shutdown.

Workflow function "adapter" applies an adapted model to an input. To use this workfl...

A lambda function that can be run within a Workflow .

Workflow function "ensemble" accepts a list of Ensembleable outputs and merges them ...

Workflow function "functionsApply" accepts a list of functions and an input and appl...

Workflow function "id" accepts a single argument and returns the result of evaluatin...

An internal WorkflowFunction that is used to execute a WorkerPoolConfig (typically a...