Inference

The inference SDK allows you to run model predictions on datasets with full control over batch processing and job management.

# Create an inference job
inference_job = client.inference(
    dataset=dataset,
    model_name="my-trained-model",
    batch_size=1000
)

# Run and wait for completion
inference_job.run(wait=True, poll_interval=10)

# Get predictions
predictions = inference_job.predictions()

inference_job.run(wait=True, poll_interval=10)

inference_job.run(wait=False)
# Check status anytime
status = inference_job.status()

Best Practices

Troubleshooting

Configuration & Model Selection Errors

ValidationError: Field required: dataset

dataset was not passed or is None. Upload or reuse the inference dataset and pass it to client.inference():

dataset = client.upload_dataframe(df_to_score, name="inference_data")
print(dataset.id)  # confirm not None
inference_job = client.inference(dataset=dataset, model_name="my_model", batch_size=1000)

Field required: model_name / model_name must be a valid string

model_name is missing or None. Use the exact model name returned from a completed training/tuning job or listed by get_all_models():

models = client.get_all_models()
print(models)  # copy the exact model name

Input should be greater than 0 / batch_size must be positive

batch_size is 0, negative, a float, or a string. Use a positive integer:

inference_job = client.inference(dataset=dataset, model_name="my_model", batch_size=1000)

Model not found / training_config.json not found / model.pt not found / 404

Fount cannot locate model artifacts for the requested model_name. The model may be from a failed/incomplete job or from a different project.

Fix: Use a completed model from get_all_models() and copy the exact name:

models = client.get_all_models()
# Verify the chosen model's training job has status Completed before running inference

Using tuned trial path directly causes artifact/config not found or state_dict mismatch

A tuned trial path was used instead of the official model name returned by the SDK. Always use the model_name returned from the completed tuning job, not a manually copied internal path.

Schema Mismatch Errors (Feature Count)

These errors all share the same root cause: the inference dataset has a different number of model input features than the dataset used during training.

size mismatch for output.0.weight

The model checkpoint shape does not match the inference input. Common triggers:

• An extra column present in inference but not in training (e.g. a row_id, prediction, or created_at column)
• A training feature column missing or renamed in inference
• Column case or whitespace changed between training and inference

Fix: Build inference data from a strict whitelist of training feature columns:

# Keep only columns that were present as features during training
training_feature_cols = ["col_a", "col_b", "Region", "Month"]  # example
df_inference = df_inference[training_feature_cols]
dataset = client.upload_dataframe(df_inference, name="inference_data")

Missing key(s) in state_dict / Unexpected key(s) in state_dict

Wrong model_name selected, or the model is from a different architecture/schema. Select model_name from the SDK model list — never edit model paths manually.

mat1 and mat2 shapes cannot be multiplied / shape mismatch during forward pass

Input tensor shape does not match the model. Schema-match the inference data exactly to the training config (same feature and target column layout).

Column case mismatch causing missing categorical columns or feature mismatch

Fount column matching is exact: Sales, sales, and Sales (with a trailing space) are treated as different columns.

Fix: Normalise column names consistently between training and inference:

print([repr(c) for c in df_inference.columns])  # spot hidden spaces
df_inference.columns = df_inference.columns.str.strip()

unexpected additional numerical columns / current model has fewer input columns than checkpoint

Extra columns were added or a training feature was dropped in the inference dataset. Remove columns that were not part of the training feature set, and restore any that were removed.

Categorical Schema Errors

Categorical columns [...] not found in the data

The inference dataset is missing categorical columns that the model was trained on. Add or rename the columns exactly as used during training:

# Compare
print(set(training_categorical_cols) - set(df_inference.columns))

ValueError: y contains previously unseen labels: '...'

A categorical value in inference was not present in training data. Options:

• Map the new category to an existing training category
• Standardise spelling/case to match training values
• Retrain the model including the new category

Label encoder for column '...' not found. Available encoders: [...]

The model artifacts contain no encoder for a column in the inference config. Use the model trained on the same categorical schema, or retrain with the intended columns.

index out of range in self

A categorical encoded value exceeds the embedding range the model was trained with. Never manually encode categorical variables before inference, let Fount handle encoding. Also check that model_name matches the dataset schema.

Numeric Data Errors

can't convert np.ndarray of type numpy.object_ / could not convert string to float

A numeric feature column contains text, currency symbols, commas, or blanks. Convert numeric features before uploading:

df_inference["Revenue"] = pd.to_numeric(
    df_inference["Revenue"].str.replace(",", "").str.replace("$", ""),
    errors="coerce"
)

Predictions contain NaN / input contains NaN or infinity

Inference inputs have missing or infinite values. Impute before uploading:

print(df_inference[numeric_cols].isna().sum())
df_inference[numeric_cols] = df_inference[numeric_cols].fillna(df_inference[numeric_cols].median())

Multi-Target & Dataset Errors

Partial target columns present. Will predict only for: [...] / Prediction columns missing for expected target

In a multi-target model, the inference dataset contains only some of the training target columns. For full multi-target predictions, either remove all target columns from inference data or include them all consistently:

# Remove all training targets from inference data
df_inference = df_inference.drop(columns=training_target_cols, errors="ignore")

Inference data loaded but predictions row count does not match expectation / empty predictions

The wrong dataset was passed (e.g. the training dataset was reused by mistake), or the dataset has zero rows. Verify the dataset before uploading:

print(df_inference.shape)   # must have rows > 0
print(df_inference.head())  # confirm it is the scoring data, not training data

Inference succeeds but predictions are misaligned with rows

Row order matters for matching predictions back to source records. Preserve row order from upload through to predictions retrieval. If you need a join key, add a stable ID column to the inference data before upload, but only if that column was not present as a training feature (to avoid the size mismatch errors above).

Performance & Memory Errors

CUDA out of memory / DefaultCPUAllocator: not enough memory / Kernel died during inference

batch_size is too large for the available memory. Reduce it:

# Start small and increase only after successful inference
inference_job = client.inference(dataset=dataset, model_name="my_model", batch_size=64)

Results Errors

Predictions not found / test_predictions.csv not found / job status Failed

Predictions are only available after a successfully completed inference job. Check status before calling predictions():

inference_job.run(wait=True)
status = inference_job.status()
if status["status"] == "Completed":
    predictions = inference_job.predictions()
else:
    print("Inference failed:", status)