Class PaddlePredictor

Nested Relationships

Class Documentation

class paddle::PaddlePredictor

A Predictor for executing inference on a model. Base class for AnalysisPredictor and NativePaddlePredictor.

Public Functions

PaddlePredictor() = default
PaddlePredictor(const PaddlePredictor&) = delete
PaddlePredictor &operator=(const PaddlePredictor&) = delete
bool Run(const std::vector<PaddleTensor> &inputs, std::vector<PaddleTensor> *output_data, int batch_size = -1) = 0

This interface takes input and runs the network. There are redundant copies of data between hosts in this operation, so it is more recommended to use the zecopyrun interface.

Return

Whether the run is successful

Parameters

  • [in] inputs: An list of PaddleTensor as the input to the network.

  • [out] output_data: Pointer to the tensor list, which holds the output paddletensor

  • [in] batch_size: This setting has been discarded and can be ignored.

std::vector<std::string> GetInputNames()

Used to get the name of the network input. Be inherited by AnalysisPredictor, Only used in ZeroCopy scenarios.

Return

Input tensor names.

std::map<std::string, std::vector<int64_t>> GetInputTensorShape()

Get the input shape of the model.

Return

A map contains all the input names and shape defined in the model.

std::vector<std::string> GetOutputNames()

Used to get the name of the network output. Be inherited by AnalysisPredictor, Only used in ZeroCopy scenarios.

Return

Output tensor names.

std::unique_ptr<ZeroCopyTensor> GetInputTensor(const std::string &name)

Get the input ZeroCopyTensor by name. Be inherited by AnalysisPredictor, Only used in ZeroCopy scenarios. The name is obtained from the GetInputNames() interface.

Return

Return the corresponding input ZeroCopyTensor.

Parameters

  • name: The input tensor name.

std::unique_ptr<ZeroCopyTensor> GetOutputTensor(const std::string &name)

Get the output ZeroCopyTensor by name. Be inherited by AnalysisPredictor, Only used in ZeroCopy scenarios. The name is obtained from the GetOutputNames() interface.

Return

Return the corresponding output ZeroCopyTensor.

Parameters

  • name: The output tensor name.

bool ZeroCopyRun()

Run the network with zero-copied inputs and outputs. Be inherited by AnalysisPredictor and only used in ZeroCopy scenarios. This will save the IO copy for transfering inputs and outputs to predictor workspace and get some performance improvement. To use it, one should call the AnalysisConfig.SwitchUseFeedFetchOp(true) and then use the GetInputTensor and GetOutputTensor to directly write or read the input/output tensors.

Return

Whether the run is successful

std::unique_ptr<PaddlePredictor> Clone() = 0

Clone an existing predictor When using clone, the same network will be created, and the parameters between them are shared.

Return

unique_ptr which contains the pointer of predictor

~PaddlePredictor() = default

Destroy the Predictor.

std::string GetSerializedProgram() const
struct Config

Base class for NativeConfig and AnalysisConfig.

Subclassed by paddle::NativeConfig

Public Members

std::string model_dir

path to the model directory.