I’m in the process of creating an MLOps workflow utilizing AWS Sagemaker pipelines with the step decorator method, and my team is currently evaluating this setup for effectively managing our machine learning operations.
We’ve put together an end-to-end pipeline that takes care of data collection, feature scaling, model training using scikit-learn’s regression methods, and generating evaluation reports. Everything functions correctly until we reach the last step, which is deployment.
The issue arises when we attempt to register and deploy our trained model using the ModelBuilder class from Sagemaker. Below is the deployment function I’ve defined:
@step(
name="deploy_model",
instance_type=compute_instance,
keep_alive_period_in_seconds=300,
)
def deploy_model(execution_path, trained_model, model_file_path, metrics_path, approval_status, validation_data_path):
import json
import numpy as np
import pandas as pd
from pathlib import Path
from sagemaker import MetricsSource, ModelMetrics
from sagemaker.serve.builder.model_builder import ModelBuilder
from sagemaker.serve.builder.schema_builder import SchemaBuilder
from sagemaker.serve.spec.inference_spec import InferenceSpec
from sklearn.linear_model import Ridge
import pickle
from s3fs import S3FileSystem
fs = S3FileSystem()
class CustomInferenceSpec(InferenceSpec):
def load(self, model_directory: str):
print(f"Loading from: {model_directory}")
loaded_model = pickle.load(fs.open(model_directory + "/trained_model.pkl", 'rb'))
return loaded_model
def invoke(self, input_data: object, model_instance: object):
results = model_instance.predict(input_data)
return results
# Setup model metrics from evaluation results
metrics = ModelMetrics(
model_statistics=MetricsSource(
s3_uri=metrics_path,
content_type="application/json",
)
)
# Prepare sample data for schema inference
feature_cols = ['feature_a', 'feature_b', 'feature_c', 'feature_d']
target_col = 'target_value'
sample_df = pd.read_csv(validation_data_path, nrows=100)
sample_df.drop(columns=[target_col], inplace=True)
schema = SchemaBuilder(
sample_input=sample_df[feature_cols].to_numpy(),
sample_output=trained_model.predict(sample_df[feature_cols]),
)
# Save model locally
local_model_dir = Path("/tmp/saved_model/")
local_model_dir.mkdir(parents=True, exist_ok=True)
with open(f"{local_model_dir}/trained_model.pkl", 'wb') as file:
pickle.dump(trained_model, file)
# Configure ModelBuilder
artifacts_path = f"{execution_path}/model_registry/artifacts"
builder = ModelBuilder(
model_path=str(local_model_dir),
inference_spec=CustomInferenceSpec(),
schema_builder=schema,
role_arn=execution_role,
s3_model_data_url=artifacts_path,
image_uri="141502667606.dkr.ecr.eu-west-1.amazonaws.com/sagemaker-scikit-learn:0.23-1-cpu-py3",
)
# Register the model
package = builder.build().register(
model_package_group_name=package_group_name,
approval_status=approval_status,
model_metrics=metrics,
)
return package.model_package_arn
I’m facing a perplexing error that states, “Can only set one of the following: model, inference_spec” despite only using the inference_spec parameter. When I remove the inference_spec, an error about missing required parameters shows up instead.
Has anyone else successfully deployed custom scikit-learn models using Sagemaker ModelBuilder? What might be amiss in my setup?