Signals: designing feature definitions that actually describe user behaviour

Machine learning systems live and die by the quality of their features. No amount of fancy architecture can save a system that is looking at the wrong input data. Conversely, an engineer who understands what drives user behaviour can get great results from a few features and a simple model.

In this post, I’ll go over some approaches to choosing the right signals, and how to engineer your features so that they model real-life relationships in your data.

What is a signal in feature engineering?

A signal is a type of machine learning feature that describes a relationship in your data. Typically it describes a real-life phenomenon that is backed by an approximation in your data.

A feature is a general term in machine learning and data science to describe a property. A signal is more specific. It describes a meaningful relationship in your data that a model should learn from.

Signals are important because machine learning models only understand numbers. Engineers building machine learning systems need to convert raw data – item names, descriptions, prices, timestamps, interaction logs into numeric inputs a model can learn from. This process of conversion is sometimes called feature engineering.

In recommendation systems, signals bridge the gap between your mental model of what drives user behaviour, and the data you have to work with.

What are some examples of signals?

Signals can be grouped into a few broad categories, each describing a different dimension of user behaviour:

• Recency: how new or fresh something is
• Popularity: how widely or frequently something is consumed
• Trendiness: whether something is gaining momentum right now
• Similarity: how closely related two items are, or how well an item matches a user’s taste
• User features: how old an account is, how much a user typically orders
• Item features: what type of product it is, availability, discount status

Once you’ve identified a category, you can calculate the signal in several ways:

• Raw column value
• Mathematical transform, like log1p, sqrt, abs, ratio
• Aggregation, like windowed count, sum, or average grouped by a key
• Bucketing continuous values into discrete categories
• Time since an event like a purchase, view, rating, etc
• Geographic distance between a user’s address and an item
• Cross product between two other features
• Similarity between user’s history embedding and an item embedding

Which combinations actually matter depend entirely on your domain. This is why having a clear mental model of your user behaviour comes in.

Deriving the right signals for your recommendation system

Consider Helen, an engineer at Woofsy, the e-commerce platform for dogs. She wants to build a recommendation system that can show users products that they may want to buy in the sidebar of her website.

Helen’s mental model of user behaviour

When approaching this problem, she already has an estimation of which features will drive clicks. She believes that users will engage with:

1. Products similar to what they are currently viewing
2. Products like what they have bought in the past
3. Popular items will do better than the best match
4. Popularity can be all-time or recent (trendiness)
5. New items should be boosted, since it’s unlikely they will have a lot of purchase history

Each of these beliefs maps onto a signal that she can derive from the item’s features. These are:

• Similarity to past purchases: cross-product of item vector and user’s purchases
• Similarity to current item: cross-product of currently viewed item and other items in the catalog
• Popularity: Purchases in the last 60 days, purchase rank among other items
• Trendiness: Purchases in the last 14 days, views in the last 14 days
• Recency: Days since product was introduced, whether it is new in the last 7 days

Notice that for each category, Helen has chosen two different signals to represent it. She can build them both and use feature importance to decide which one drives results the best.

Using a mental model of user behaviour to build better recommendations

It can be easy to calculate a lot of different signals and throw them at a model, but without a mental model that describes the actual behaviour of your users, you won’t know what to experiment with.

Here are some sample questions to ask yourself, based on your vertical:

E-Commerce

• Do users buy multiple products in one order?
• Do users re-buy the same products, or explore?
• Are users price sensitive?
• Does seasonality affect what people buy?
• Are there products that get viewed but rarely bought?

Content (video, audio, social)

• Do users binge in one sitting, or graze across sessions?
• Is engagement driven more by the creator or topic?
• How quickly does content go stale?
• Do users finish what they start?
• Does time of day affect what people want?

Publishing, News, and Editorial

• Do readers come for specific topics, or browse broadly?
• Are users driven by recency, or depth?
• Do users follow specific authors?
• Is there a meaningful difference between subscribers and anonymous visitors?
• Do users arrive with specific intent, or browse organically?

Common problems in feature engineering

When defining signals, it may be tempting to throw everything at the wall to see what sticks. But this approach has its limitations.

Interpretability: if your results are good, do you know why? Without a deliberate set of signals, you can’t tell which features are driving performance, which makes it hard to improve further or explain results to stakeholders.

Knowing what to drop: if results are bad, what do you change? A bloated feature set makes it hard to isolate what’s not working. Starting from a mental model means you have a principled reason to add or remove a signal.

Feature drift: after six months in production, someone changes how an aggregation is defined, or a data pipeline shifts. Signals that were meaningful when you built them quietly stop being meaningful. Keeping your signal definitions explicit and documented makes drift easier to catch.

Pipeline complexity: every signal you add is something to compute, maintain, and monitor. A simpler set of well-chosen signals is almost always easier to operate than a large set of loosely motivated ones.

Missing data: many teams don’t have embeddings for their items because building and maintaining that pipeline is non-trivial. It’s worth knowing which of your signals depend on data you don’t reliably have.

---

The good news is that a lot of this complexity is manageable if you treat signal definition as a first-class part of your system design — not an afterthought.

At Shaped, we built the Signal Engine specifically to address this. Rather than writing feature pipelines by hand, you define your signals declaratively and Shaped handles the computation, storage, and serving. It’s designed to make the gap between “I have a mental model of my users” and “I have a working recommendation system” as small as possible.