From Unified Event Streams to Intelligent Interactions

Rudderstack is a powerful open-source Customer Data Platform (CDP) designed for developers, enabling businesses to collect event data from various sources (website, mobile apps, servers) and route it reliably to different destinations, including data warehouses and real-time tools. This unified event stream provides a comprehensive picture of user interactions across your digital properties. While Rudderstack excels at collecting and routing this data, the next crucial step is activating it – transforming the stream into intelligent, personalized experiences as events happen.

How do you leverage the unified view of user behavior flowing through Rudderstack to instantly tailor recommendations or search results? How do you apply sophisticated AI models to this real-time stream without building and managing complex stream processing and machine learning infrastructure? This is where Shaped's direct integration with Rudderstack via AWS Kinesis provides a seamless and potent solution.

Shaped is an AI-native relevance platform architected to ingest real-time event streams, like those managed by Rudderstack, train state-of-the-art machine learning models, and serve personalized search rankings and recommendations through simple APIs. This post explains the benefits of connecting these platforms and provides a step-by-step guide using the AWS Kinesis destination.

Why Connect Rudderstack to Shaped? Powering Use Cases with Unified Data

Sending your Rudderstack event stream directly to Shaped via Kinesis unlocks sophisticated AI capabilities, turning your data routing hub into an engine for real-time relevance:

• Hyper-Personalized Recommendations: Utilize the rich, unified behavioral data collected by Rudderstack from all your sources:
  
  • Truly Omnichannel Personalization: Generate recommendations informed by a user's complete journey across web, mobile, and server-side events unified by Rudderstack.
  • Dynamic "For You" Feeds: Curate highly relevant content or product feeds that adapt instantly based on the latest events streamed from Rudderstack.
  • Context-Aware Suggestions: Recommend related items or next actions based on real-time session activity captured across different sources via Rudderstack.
• Intelligent & Personalized Search: Enhance search relevance using the live behavioral signals flowing through Rudderstack:
  
  • Real-Time Personalized Ranking: Adjust search result order based on immediate user interactions (clicks, views, adds-to-cart) captured by Rudderstack sources.
  • Behaviorally-Informed Search: Use the patterns learned from the Rudderstack event stream to improve baseline search relevance.
• Advanced Analytics & Insights: Apply Shaped's ML models to the unified Rudderstack stream for deeper understanding:
  
  • Cross-Source Journey Analysis: Model complex user paths and predict behavior based on events collected from multiple platforms via Rudderstack.
  • Real-Time Audience Segmentation: Use user embeddings generated by Shaped based on live Rudderstack data for dynamic targeting or analysis.
• Real-Time Adaptability: This Kinesis-based integration ensures Shaped's models learn and adapt almost instantaneously as new events flow from Rudderstack, keeping personalization hyper-relevant.
• Simplified ML Infrastructure: Avoid the complexity of building custom stream processing consumers, ML training pipelines, and serving infrastructure for your Rudderstack data. Shaped provides the managed AI layer specifically designed for real-time relevance.

How it Works: Rudderstack -> Kinesis -> Shaped

This integration leverages AWS Kinesis Data Streams as the secure and scalable real-time bridge:

1. Shaped Provisions Kinesis: You create a RUDDERSTACK schema type dataset in Shaped. Shaped automatically provisions a dedicated Kinesis Data Stream configured for the standard Rudderstack event format and provides secure access credentials.
2. Shaped Provides Credentials: Shaped gives you the ARN of the Kinesis stream and an IAM Role ARN specifically designed for Rudderstack's Kinesis destination to use.
3. Rudderstack Sends to Kinesis: You configure the "Amazon Kinesis" destination within your Rudderstack dashboard, providing the Stream Name and IAM Role ARN supplied by Shaped. Rudderstack then securely streams event data to Shaped's Kinesis endpoint.
4. Shaped Ingests & Learns: Shaped continuously reads events from the Kinesis stream, updates its AI models in near real-time, and serves personalized results via its APIs.

Connecting Rudderstack to Shaped via Kinesis

The process involves setting up the endpoint in Shaped first, then configuring the Kinesis destination in Rudderstack.

Step 1: Create the Shaped Dataset

First, create a dataset in Shaped specifically configured to receive Rudderstack events. Using schema_type: RUDDERSTACK ensures Shaped automatically understands the standard Rudderstack event structure.

1. Define the Dataset (YAML): Create a YAML file. Note that you must include your AWS Account ID in tenant_aws_account_id to allow Rudderstack's Kinesis destination (running within your infrastructure or Rudderstack's cloud) to assume the role Shaped creates.

1 name: rudderstack_events
2 schema_type: RUDDERSTACK
3 
4 # IMPORTANT: Provide your AWS Account ID
5 # where your Rudderstack infra runs (or Rudderstack's account)
6 tenant_aws_account_id: "YOUR_AWS_ACCOUNT_ID"
7 
8 # 'column_schema' not required — inferred from Rudderstack standard schema
9 # 'unique_keys' like 'messageId' are handled automatically by Shaped
10 # schedule_interval: "@hourly" — Kinesis is real-time

2. Create the Dataset: Use the Shaped CLI. The underlying Kinesis stream provisioning takes a few minutes.

1 shaped create-dataset --file rudderstack_events.yaml

3. Monitor Status: Wait for the dataset status to become ACTIVE using the Shaped Dashboard or CLI (shaped view-dataset --dataset-name rudderstack_events).

Step 2: Retrieve Shaped Kinesis Details

Once the dataset is ACTIVE, Shaped provides the necessary details for the Rudderstack configuration. Retrieve these via CLI, API, or Dashboard:

• Kinesis Stream ARN: The full ARN of the stream Shaped created (e.g., arn:aws:kinesis:us-east-2:11111111111:stream/ShapedDatasetStream-abc123). You'll need the Stream Name part.
• Kinesis Access Role ARN: The ARN of the IAM Role created by Shaped that Rudderstack's Kinesis destination needs to assume (e.g., arn:aws:iam::11111111111:role/ShapedDatasetAccessRole-abc123). You'll need this full ARN.

Note down the Stream Name and the full Access Role ARN.

Step 3: Configure Rudderstack Kinesis Destination

Now, configure Rudderstack to send data to the stream Shaped provisioned.

1. Log in to your Rudderstack Dashboard.
2. Navigate to Destinations > Add Destination.
3. Search for and select the Amazon Kinesis destination type.
4. Give the destination a Name (e.g., "Shaped Production Stream").
5. Click Next.
6. Enter the following connection details obtained from Shaped in Step 2:
   
   • AWS Region: Enter us-east-2 (Shaped provisions Kinesis streams in this region for these integrations).
   • AWS Kinesis Stream Name: Enter the name extracted from the Kinesis Stream ARN (e.g., ShapedDatasetStream-abc123).
   • AWS IAM Role Resource Name (ARN): Enter the full Kinesis Access Role ARN provided by Shaped.
7. Click Next.
8. (Crucial) Connect Sources: Go to the Sources tab for this destination and select the Rudderstack Source(s) that contain the event data you want to send to Shaped for personalization.
9. (Optional but Recommended) Transformation: Consider using Rudderstack Transformations if you need to filter specific events before they are sent to Kinesis/Shaped, although sending key engagement events is generally recommended.
10. Activate: Ensure the destination is enabled/activated.

Refer to the official Rudderstack documentation for the Amazon Kinesis destination for the most current UI walkthrough and specific configuration options within their platform.

What Happens Next? Streaming Unified Data to AI Models

With the Rudderstack Kinesis destination active:

1. Rudderstack Routes Events: As events arrive from your connected sources, Rudderstack routes them to the configured Kinesis destination.
2. Shaped Ingests: Shaped reads the events from the Kinesis stream in near real-time.
3. Real-Time Learning: Shaped's AI models process these unified events, updating user profiles and personalization logic almost instantly.
4. Personalized API Responses: Shaped's APIs serve recommendations and search rankings reflecting the latest insights derived from the Rudderstack event stream.

Conclusion: Activate Your Unified Customer Data with Rudderstack and Shaped

Rudderstack provides a powerful foundation for collecting and routing customer event data across your entire stack. By connecting Rudderstack directly to Shaped via the Kinesis destination, you seamlessly bridge this unified data stream with a state-of-the-art AI relevance engine. This integration allows you to move beyond simple data routing and activate your valuable event data to power truly personalized, real-time recommendations and search experiences, driving deeper engagement and better business outcomes.

Ready to turn your Rudderstack event stream into intelligent personalization?

Request a demo of Shaped today to see it in action with your specific use case. Or, start exploring immediately with our free trial sandbox.